jaithree/grug-9b-mlx-q4
Text Generation • 1B • Updated • 18
jaithree
jaithree
AI & ML interests
"In the beginning was the word. Then came the **** word processor. Then came the thought processor. Then came the death of literature. And so it goes" -- Hyperion - Dan Simmons
Recent Activity
liked a model about 7 hours ago
ReadyArt/gemma-4-31B-it-scotoma-GGUF liked a model about 7 hours ago
Ateron/Gemma-4-Novelist-Eclipse-31B reacted to mike-ravkine's post with 🔥 about 12 hours ago
I went into this expecting to find a ~Q2 garbage dumpster but https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf is a slick feat of QAT engineering.
It is weaker then FP16 on a handful of tasks where quants usually degrade, as per their own paper the loss is "concentrated on sustained chains of reasoning / agentic" and in ReasonScape this bites on Sort, Shuffle and Dates, but counter-acting this are some noticeable improvements to thinking length without accuracy loss on several other tasks (Shapes, Cars).
I haven't had a chance to run the Binary yet, but PQ2 + Bonsai QAT are confirmed to be pretty darn impressive. Organizations
None yet