Update README.md
Browse files
README.md
CHANGED
|
@@ -5,14 +5,18 @@ license: other
|
|
| 5 |
license_name: cc-by-nc-4.0-weights-apache-2.0-code
|
| 6 |
license_link: LICENSE.md
|
| 7 |
tags:
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
- causal-lm
|
| 14 |
language:
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
# Three-Level Nested Byte Mamba-2
|
|
@@ -33,6 +37,7 @@ The published weights are sharded SafeTensors containing only the 980 model tens
|
|
| 33 |
- Source checkpoint step: **889,000**
|
| 34 |
- Recommended runtime precision: **BF16**
|
| 35 |
- Recommended placement: fine/decoder on `cuda:0`, level 2 on `cuda:1`, level 3 on `cuda:2`
|
|
|
|
| 36 |
|
| 37 |
The source checkpoint step is documentation only; it is not embedded in the SafeTensors weights or inference configuration.
|
| 38 |
|
|
@@ -207,4 +212,4 @@ datasets and third-party content are not covered by either grant. See
|
|
| 207 |
- `nested_inference_tools.py`: SafeTensors loading, state caching, and sampling
|
| 208 |
- `infer_nested_model.py`: command-line generator
|
| 209 |
- `architecture.svg`: architecture visualization
|
| 210 |
-
- `LICENSE.md`: weight, documentation, and code license scope
|
|
|
|
| 5 |
license_name: cc-by-nc-4.0-weights-apache-2.0-code
|
| 6 |
license_link: LICENSE.md
|
| 7 |
tags:
|
| 8 |
+
- mamba2
|
| 9 |
+
- byte-level
|
| 10 |
+
- multimodal
|
| 11 |
+
- custom-code
|
| 12 |
+
- base_model
|
|
|
|
| 13 |
language:
|
| 14 |
+
- en
|
| 15 |
+
datasets:
|
| 16 |
+
- HuggingFaceM4/FineVision
|
| 17 |
+
- HuggingFaceFW/finepdfs
|
| 18 |
+
- HuggingFaceFW/fineweb-edu
|
| 19 |
+
- webshart/suno-various-94k
|
| 20 |
---
|
| 21 |
|
| 22 |
# Three-Level Nested Byte Mamba-2
|
|
|
|
| 37 |
- Source checkpoint step: **889,000**
|
| 38 |
- Recommended runtime precision: **BF16**
|
| 39 |
- Recommended placement: fine/decoder on `cuda:0`, level 2 on `cuda:1`, level 3 on `cuda:2`
|
| 40 |
+
- Last training count: 15GB **Absoloutly undertrained**
|
| 41 |
|
| 42 |
The source checkpoint step is documentation only; it is not embedded in the SafeTensors weights or inference configuration.
|
| 43 |
|
|
|
|
| 212 |
- `nested_inference_tools.py`: SafeTensors loading, state caching, and sampling
|
| 213 |
- `infer_nested_model.py`: command-line generator
|
| 214 |
- `architecture.svg`: architecture visualization
|
| 215 |
+
- `LICENSE.md`: weight, documentation, and code license scope
|