Article
Train a Sentence Embedding Model with 1B Training Pairs
asi
• • 2
Renderlib-dev
Renderlib-dev
AI & ML interests
None yet
Recent Activity
upvoted an article 4 days ago
Train a Sentence Embedding Model with 1B Training Pairs updated a Space 7 days ago
Renderlib-dev/CodeVed updated a Space 10 days ago
Renderlib-dev/HelloOrganizations
upvoted an article 4 days ago
updated a
Space 7 days ago
updated a
Space 10 days ago
published a
Space 10 days ago
updated a
Space 13 days ago
published a
Space 13 days ago
Gradio demo model loading problem
2
#4 opened 17 days ago
by Renderlib-dev
reacted to Banaxi-Tech's post with 👀🔥 17 days ago
Post
4244
Today we are releasing BananaMind-KV1-8M-2Bit-Experimental, a KV-cache-aware trained model that stores its generation KV cache in 2-bit precision instead of the usual 16-bit precision.
Result: 5.33x smaller KV cache vs FP16, with 0.0916 mean KLD against a 16-bit KV cache reference on WikiText-2.
Model: BananaMind/BananaMind-KV1-8M-2Bit-Experimental
The important part: this is not just post-training KV cache quantization.
Instead we take the BitNet approach.
KV1 is trained with a 2-bit-aware K/V path. Instead of training a normal model and quantizing the cache afterwards, the model learns during training to operate under the low-bit KV constraint, closer in spirit to the BitNet idea of training for the low-bit regime.
During generation, each K/V vector is quantized into 4 affine levels and packed into uint8 tensors, with four 2-bit values stored per byte.
WikiText-2 eval vs 16-bit KV cache reference:
Mean KLD: 0.0916 nats/token
Mean KLD: 0.1322 bits/token
Average KV cache shrink vs FP16: 5.33x
Evaluated positions: 372,675
If this actually gets used in models like Qwen or Gemma, then it may be possible to run 128K or even 256K Context on a Normal Machine!
Try it here: BananaMind/BananaMind-KV1-8M-2Bit-Experimental
Code: https://github.com/Banaxi-Tech/kv1
Result: 5.33x smaller KV cache vs FP16, with 0.0916 mean KLD against a 16-bit KV cache reference on WikiText-2.
Model: BananaMind/BananaMind-KV1-8M-2Bit-Experimental
The important part: this is not just post-training KV cache quantization.
Instead we take the BitNet approach.
KV1 is trained with a 2-bit-aware K/V path. Instead of training a normal model and quantizing the cache afterwards, the model learns during training to operate under the low-bit KV constraint, closer in spirit to the BitNet idea of training for the low-bit regime.
During generation, each K/V vector is quantized into 4 affine levels and packed into uint8 tensors, with four 2-bit values stored per byte.
WikiText-2 eval vs 16-bit KV cache reference:
Mean KLD: 0.0916 nats/token
Mean KLD: 0.1322 bits/token
Average KV cache shrink vs FP16: 5.33x
Evaluated positions: 372,675
If this actually gets used in models like Qwen or Gemma, then it may be possible to run 128K or even 256K Context on a Normal Machine!
Try it here: BananaMind/BananaMind-KV1-8M-2Bit-Experimental
Code: https://github.com/Banaxi-Tech/kv1
- 4 replies
bharatgenai/sooktam2
🚀➕ 20
7
#8550 opened 4 months ago
by 
yagnikt
liked a
model 18 days ago
liked a
Space 18 days ago
liked a
Space about 1 month ago
Update index.html
#1 opened about 1 month ago
by 
Hanuman2
liked a
Space about 1 month ago
reacted to danielhanchen's post with 👍👍 about 2 months ago
Post
9328
Gemma 4 12B can now run locally on just 8GB RAM via Dynamic GGUFs.
Google's new model, Gemma 4 12B Unified supports image, audio and 256K context.
You can run and train the model via Unsloth Studio.
GGUF: unsloth/gemma-4-12b-it-GGUF
Guide: https://unsloth.ai/docs/models/gemma-4
Google's new model, Gemma 4 12B Unified supports image, audio and 256K context.
You can run and train the model via Unsloth Studio.
GGUF: unsloth/gemma-4-12b-it-GGUF
Guide: https://unsloth.ai/docs/models/gemma-4
- 5 replies
liked a
Space about 2 months ago