PrimeIntellect/Laguna-XS.2-jlens

Laguna-XS-2.1 J-lens: bf16 vs NVFP4 A/B

Two Jacobian lenses of poolside/Laguna-XS-2.1 (~33B fine-grained MoE, 40 layers, d=2048, 256 experts/8 active) fitted on byte-identical prompts (200 × wikitext-103, seq 128, all 39 source layers, target 39) — one from the bf16 checkpoint, one from the official NVFP4 twin. The twins' non-expert tensors are bit-identical (verified: lm_head, final norm, embeddings, attention, router), so every lens difference is attributable to NVFP4 quantization of expert weights (plus one ~0.6% export-time perturbation of the 3 layer-0 dense-MLP tensors, documented in the report).

Files

• laguna_bf16_jacobian_lens.pt, laguna_nvfp4_jacobian_lens.pt — merged n=200 lenses (fp16 payload; fp16 rounding is irrelevant at 1−CKA ≈ 2e-9)
• shards_{bf16,nvfp4}/ — 8 per-GPU shard lenses per arm (fp32, 25 prompts each; shard i = same prompts in both arms)
• convergence_{bf16,nvfp4}/ — per-shard convergence CSVs
• results/ — ab_summary.json, per-analysis npz, figures, REPORT.md

Probe: P = W_U[ids]·γ (plain-γ RMSNorm, no softcap), ids = 4096 frozen seed-0 over vocab 100352; identical across arms. Fit code: open-jlens-data repo (code/moe/fit_moe.py laguna dispatch, nvfp4_experts.py, analyze_laguna_ab.py). Fitted + analyzed 2026-07-17. Context: this A/B calibrates the NVFP4 caveat of the Inkling ~950B lens (PrimeIntellect/inkling-jlens) — same modelopt recipe family; transfer caveat: Laguna experts are 512-dim vs Inkling 3072-dim.

xs2/ — Laguna-XS.2 version-drift extension (E9)

xs2/ holds the same fit for poolside/Laguna-XS.2 (bf16, identical 200 prompts/probe/tokenizer) plus the version comparison vs XS-2.1. Verdict: version drift is a regime change — same-layer CKA median 0.638 = 43× the NVFP4 effect and 54× prompt noise — while the depth architecture survives (both versions k=2 at layer 27/28, cross-matrix depth correlation 0.990). The versions share no token basis (same-token unembedding rows near-orthogonal), so each is read through its own probe. Perturbation ladder on identical prompts (1−CKA): noise 0.007 < NVFP4 0.008 ≪ version 0.362.