leuconoe/litert-lm-unity-quantized

Community Release — Project-Quantized LiteRT Models (2026-07-23)

Quantized ASR/LLM artifacts produced by the LiteRT-LM-Unity project, prepared for upstream contribution (e.g., litert-community on Hugging Face). All files were validated before inclusion; validation data below.

Toolchain / provenance

• Quantizer: ai-edge-quantizer 0.8.0 (post-training, dynamic-range).
• Recipes:
  • i8 = recipe.dynamic_wi8_afp32() (int8 weights channelwise, fp32 activations).
  • i4 = recipe.dynamic_wi4b64_afp32() (int4 weights, blockwise-64, fp16 scales) with i8 overrides on sensitive scopes (encoder / embedding-logits table / selected projections). Pure full-scope int4 was tested and REJECTED for several models (Korean transcription errors); the mixed recipes below are what passed.
  • int4 channelwise (wi4c) and blockwise-32 (wi4b32) are known-bad (quality collapse / immediate-EOS respectively) — do not regenerate with them.
• Whisper medium / large-v3 / large-v3-turbo tflite graphs were exported by this project from the openai checkpoints (30 s fixed window, encode/decode split signatures matching the litert-community whisper-tiny/base graph interface; large-v3 family = 128 mel bins / 51866 vocab). f32 source exports (3–6 GB) are retained separately and available on request.
• Qwen2.5 .litertlm files were produced by unpack → quantize → repack: litert-lm-peek dump → dynamic_wi4b64_afp32 on the f32 tflite section → rebuild with litert-lm-builder 0.14.0 (tokenizer + LlmMetadata mirrored from the official q8 release). Source: official litert-community f32 releases.
• Runtime compatibility: .litertlm format 1.5.0; validated on LiteRT-LM v0.14.0 (Windows x86_64 CPU/GPU and Android arm64, Snapdragon 865-class device).

Files and validation

Validation clips: Korean "2025년 3월 5일 전술평가 결과 보고" sentence + English equivalent + short Korean voice commands ("볼륨 업" etc.). CER computed against reference transcripts (punctuation-normalized). Full matrix: docs/benchmarks/asr-model-matrix.md in the LiteRT-LM-Unity repo.

File	Size	Base model	Recipe	Validation
whisper-tiny/whisper_tiny_30s_i4.tflite	35 MB	openai/whisper-tiny (via litert-community f32)	wi4b64 + i8 encoder+embed table	ko/en gate clips exact
whisper-base/whisper_base_30s_i8.tflite	74 MB	openai/whisper-base (via litert-community f32)	wi8	transcripts byte-identical to f32; ~1.6× faster
whisper-base/whisper_base_30s_i4.tflite	44 MB	same	wi4b64 (full scope OK for base)	ko/en exact
whisper-medium/whisper_medium_30s_i8.tflite	794 MB	openai/whisper-medium (project export)	wi8 (DRQ)	7/9 exact, CER ko 0.042 / en 0.000
whisper-medium/whisper_medium_30s_i4.tflite	634 MB	same	wi4b64 mixed (L1)	same transcripts as i8
whisper-large-v3/whisper_large_v3_30s_i8.tflite	1.6 GB	openai/whisper-large-v3 (project export, 128-mel)	wi8 (DRQ)	CER 0.000/0.000, 7/9 exact (spacing-only misses)
whisper-large-v3/whisper_large_v3_30s_i4.tflite	1.1 GB	same	wi4b64 mixed (mixD)	CER 0.000/0.000; ~20 % faster than i8
whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i8.tflite	1.1 GB	openai/whisper-large-v3-turbo (project export, 128-mel)	wi8	5–6/9 exact, CER ko 0.000
whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i4.tflite	721 MB	same	wi4b64 mixed (mixD)	best tier overall: 8/9 exact, CER 0.000/0.000; device-validated (Snapdragon 865, shape-based decode binding)
qwen2.5-0.5b-instruct/Qwen2.5-0.5B-Instruct_wi4b64_ekv1280.litertlm	265 MB	litert-community/Qwen2.5-0.5B-Instruct f32	wi4b64 full scope	inference-validated Windows + Android (35.5 tok/s device CPU, +38 % vs official q8)
qwen2.5-1.5b-instruct/Qwen2.5-1.5B-Instruct_wi4b64_ekv4096.litertlm	790 MB	litert-community/Qwen2.5-1.5B-Instruct f32	wi4b64 full scope	inference-validated Windows CPU (59.7 prefill / 11.8 decode tok/s, Korean QA correct)

Related: ACFT short-window Whisper models

Fixed 5 s/10 s/30 s short-window TFLite graphs (encode-signature-sized mel window; method: futo-org/whisper-acft) live in two dedicated repos:

• Pure ACFT (TFLite conversions of the futo-org checkpoints, no retraining; tiny / base / small × multilingual / .en, dynamic-range int8): litert-community/whisper-acft.
• Korean ACFT (ACFT-KO, Korean-trained for on-device Korean voice commands; tiny / base / medium / large-v3-turbo, dynamic-range int8): leuconoe/whisper-acft-ko. Training data zeroth-korean (OpenSLR SLR40, CC-BY-4.0) + google/fleurs en_us (CC-BY-4.0).

Known caveats (disclose in model cards)

• Whisper 30 s graphs; decoder is fixed-length re-run (no KV cache) matching the litert-community tiny/base interface. large-v3 family needs a 128-mel frontend and vocab 51866 (<|transcribe|>=50360, <|notimestamps|>=50364).
• tiny/base i8 showed sub-1.2 s clip sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); turbo/large-v3 unaffected.
• Qwen2.5 i4 litertlm metadata carries lowercase model_type (tf_lite_prefill_decode) vs uppercase in official files — loads fine on v0.14.0, noted for completeness.
• Licenses: whisper weights MIT (OpenAI); Qwen2.5 Apache-2.0. Quantized derivatives inherit the base licenses.