Alienanthony commited on
Commit
dd25e82
·
verified ·
1 Parent(s): 479a45a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -8
README.md CHANGED
@@ -5,14 +5,18 @@ license: other
5
  license_name: cc-by-nc-4.0-weights-apache-2.0-code
6
  license_link: LICENSE.md
7
  tags:
8
- - mamba2
9
- - byte-level
10
- - state-space-model
11
- - multimodal
12
- - custom-code
13
- - causal-lm
14
  language:
15
- - en
 
 
 
 
 
16
  ---
17
 
18
  # Three-Level Nested Byte Mamba-2
@@ -33,6 +37,7 @@ The published weights are sharded SafeTensors containing only the 980 model tens
33
  - Source checkpoint step: **889,000**
34
  - Recommended runtime precision: **BF16**
35
  - Recommended placement: fine/decoder on `cuda:0`, level 2 on `cuda:1`, level 3 on `cuda:2`
 
36
 
37
  The source checkpoint step is documentation only; it is not embedded in the SafeTensors weights or inference configuration.
38
 
@@ -207,4 +212,4 @@ datasets and third-party content are not covered by either grant. See
207
  - `nested_inference_tools.py`: SafeTensors loading, state caching, and sampling
208
  - `infer_nested_model.py`: command-line generator
209
  - `architecture.svg`: architecture visualization
210
- - `LICENSE.md`: weight, documentation, and code license scope
 
5
  license_name: cc-by-nc-4.0-weights-apache-2.0-code
6
  license_link: LICENSE.md
7
  tags:
8
+ - mamba2
9
+ - byte-level
10
+ - multimodal
11
+ - custom-code
12
+ - base_model
 
13
  language:
14
+ - en
15
+ datasets:
16
+ - HuggingFaceM4/FineVision
17
+ - HuggingFaceFW/finepdfs
18
+ - HuggingFaceFW/fineweb-edu
19
+ - webshart/suno-various-94k
20
  ---
21
 
22
  # Three-Level Nested Byte Mamba-2
 
37
  - Source checkpoint step: **889,000**
38
  - Recommended runtime precision: **BF16**
39
  - Recommended placement: fine/decoder on `cuda:0`, level 2 on `cuda:1`, level 3 on `cuda:2`
40
+ - Last training count: 15GB **Absoloutly undertrained**
41
 
42
  The source checkpoint step is documentation only; it is not embedded in the SafeTensors weights or inference configuration.
43
 
 
212
  - `nested_inference_tools.py`: SafeTensors loading, state caching, and sampling
213
  - `infer_nested_model.py`: command-line generator
214
  - `architecture.svg`: architecture visualization
215
+ - `LICENSE.md`: weight, documentation, and code license scope