Magnus Lundstedt PRO

The abstain pricing Is what I keep coming back to, you turned the open problem into a lookup. Among block-quant formats it’s exactly three: (32,18), (256,66), (256,110). There is a test now that fails if a future row breaks it. You’re right the ratio would never have named MXFP4, fallback already computes the pair and then throws the identity away.

Id 40/41/42 (NVFP4, Q1_0, Q2_0) added and Q8_1 pinned, a limit, and it’s the interesting one: We know of no public model on the hub that carry any of these four types, so they are right by construction and test but not yet observation. If you have such model that’s the evidence we’re missing to verify end-to-end with it.

Q8_1, you are right and we’re right; gguf-py carrying the pre-ggml_half2 (32,40), four bytes a block, pinned id 9 to ggml.c You found their bug through our table and it should also go back to them, so we’re sending that one upstream with credit to you.

One small open thing, since you compute to six places, on the MoE merge our shipped code lands at 4.254078 / 5.8747%, close to your 4.255556 / 6.0052% but not identical, and I haven’t found the derivation that gives yours. Doesn’t change the fix (keying on the block-prefix), but if you can share how you got yours we’d rather close the gap than leave two numbers standing.

Your real question, geometry vs a declared name, which wins: The number is always the geometry, bytes are ground truth, which is why we came out right on Q8_1 and the reference didn’t, the name is the declared id, but geometry is its check and a disagreement is surfaced, not silently resolved. We already do that (the cross-check that caught a bf16 file whose tensors came out at sizes with offsets contradicted). So: name missing -> geometry names it or declines, name present and agrees -> confirmed. Name present and disagrees -> we report the conflict and size by the bytes, a silent winner either way is the confident-wrong failure this whole thing exists to avoid.

Acknowledgements, yes whenever you want in and however you’d like to be credited, say the word.

Four for four and this one found a bug in the canonical reference too, keep going! Thanks! 😄

Great report, and the byte identical 10152.. across the two containers is what makes it airtight. Same payload, not just the same ratio. Thanks for deriving against the current bundle.

Both going in now:

• Type 39 -> MXFP4. Your 32/17 is exactly what the safe tensors path already documents and 72/72 with no layout inference is the whole argument. The “undocumented” warning was just wrong here.
• Grouping by blocks on “.blocks” vs. “_blocks” grouper now fixed. Matching the leaf regardless of the separator now. It still need to pass geometry check, so a stray dot match can’t slip false pair through.

Your MoE notes is useful, the code keys off block-name prefix and is fine but the rule as currently stated “group by module path”) is what cause issue here, since experts hold two projections and tow biases and there is no expert down projection submodule. Fixing the rule here.

On your real question, honest answer no. Nowhere today does a block format get identified by geometry with the name as a hint. In both misses the geometry is computed and then used only as size, the GGUF fallback solves (blockSize, typeSize) and gets the bits right while leaving the name “type39” and the safetensor grouper geometry-verifies the pair but keys identity off the suffix. So the label owns identity and the geometry that would name it is right there doing nothing but arithmetic. That’s why the numbers were right both times and the names wrong both times, and we’re convinced now we got it backwards.

So we’ll build what you are describing, a (blockSize, typeSize) -> format signature check when id/suffix doesn’t resolve. It’s tracked in our plan and deserves some extra attention later. Your question did what the two bug alone didn’t, named one structural gap they were symptoms of. The hard part is abstaining when two formats share a geometry, corroborate or decline rather than guess.

Three for three reports now, keep breaking it! 😄 Thanks! (Btw if you want to be mentioned on our acknowledgement page (about section) we’re happy to add you!

Thanks, that's right, I wanted reports recoverable without an account or storing any identity, so a bearer-style hash over the find was the natural fit. Glad it read that way.

Agreed on surfacing parsed architecture + quant metadata beside the tensor view, that's the direction I want to take it, and it's most valuable in the comparison mode where cross-format discrepancies bite. Right now clicking a tensor gives you the sampled distribution plus dtype/bits/group-size; adding the full parsed quant metadata (format, block layout, scale placement) next to it is what turns a diff from "these two numbers differ" into "here's why" 😄

Hardest edge case so far, I would say GGUF IQ-quants (IQ1_S / IQ2_XXS, etc), the codebook/lookup block structures are tricky to get right. Or NF4 with the nested/double quantization (quantized absmax). 😅

If you hit a format that you think renders wrong, the report button is right there. Those reports genuinely help, and I especially welcome the hard ones. 🙏

Confirmed again, you are right that NF4 fails quietly, 2x wrong looks like int8 where 8x wrong looks broken. We verified your numbers, the 4.127, the exact 4.5000 without double quant, the 0.3729 bits it buys, all reproduce, and we've fixed the accounting now! Decoding NF4 values for plots comes later, its codebook belongs with a batch of similar formats we verify together.

Your grouping proposal is adopted: we now key groups off the module path and interpret per family, with anything uninterpreted staying visible as unexplained bytes, which means the next scheme like this should flag itself (we have two thresholds for highlighting unrecognized tensors for yellow and red warning at the bottom). Also taking your denominator point, the UI will state of-group and keep of-bare as a diagnostic. The double quant ordering result (codebook share tracks group granularity, not bit width) is going on the site with credit to you. Thanks for several real findings with measurements attached now. At some point we should have a proper acknowledgements page and you have earned a spot on it. 😄

Let us know if you find more challenging models! 😄

Yes, Tensorlens group qweight/qzeros/scale triples, but for these examples it grouped the two sidecars but left qweight out, reading 32 bits. So thanks for the challenging models! Turns out there were many problems to fix. 😅 Now both Qwen/Qwen2.5-7B-Instruct-AWQ and TheBloke/TinyLlama-1.1B-Chat-v0.3-GPTQ work!

Now the map will calculate correct bpw value for the group, matching your numbers. I also added your idea on how much of budget is codebook rather than weights (3.8% and 4.1%) in the summary below the map.

A challenge with AWQ was the 4-bit weights stored in a declared I32 in the safetensors header (8 × 4-bit weights), in an unusual order (0,2,4,6,1,3,5,7) as opposed to the more natural assumption that position X is for column X. And with GPTQ it was the descending activation order rather than left to right, so every feature has to be looked up in g_idx to find its group and without that the weights decode subtly wrong.

Worth flagging, there might be similar models (and many others) that still decode wrong, so if you test some other model and numbers look off, that’s another bug I am happy to hear about! 😄