Instructions to use oddadmix/50M-2048-Emhotob with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use oddadmix/50M-2048-Emhotob with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="oddadmix/50M-2048-Emhotob")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("oddadmix/50M-2048-Emhotob")
model = AutoModelForCausalLM.from_pretrained("oddadmix/50M-2048-Emhotob")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use oddadmix/50M-2048-Emhotob with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "oddadmix/50M-2048-Emhotob"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oddadmix/50M-2048-Emhotob",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/oddadmix/50M-2048-Emhotob

SGLang

How to use oddadmix/50M-2048-Emhotob with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "oddadmix/50M-2048-Emhotob" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oddadmix/50M-2048-Emhotob",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "oddadmix/50M-2048-Emhotob" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oddadmix/50M-2048-Emhotob",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use oddadmix/50M-2048-Emhotob with Docker Model Runner:
```
docker model run hf.co/oddadmix/50M-2048-Emhotob
```

50M-2048-Emhotob (إمحوتب) — Arabic Base Model

Emhotob is a ~51.8M-parameter Llama-architecture language model pre-trained from scratch on ~20 billion Arabic tokens with a 2048-token context window. It is a proof-of-concept that a tiny model — small enough to run on a CPU — can be trained from scratch to produce coherent Arabic.

This is a base (pretrained) model: a next-token predictor with no instruction or chat tuning. It is the foundation for the fine-tuned variants below.

Architecture & training recipe: derived from SupraLabs/Supra-50M-Base ("Project Chimera"), re-run from scratch on an Arabic corpus with a custom Arabic tokenizer.
Pre-training corpus: kaust-generative-ai/fineweb-edu-ar (Arabic).

الملخص بالعربية: «إمحوتب» نموذج لغوي عربي صغير (~51.8 مليون معامل) بمعمارية Llama، مُدرَّب من الصفر على ~20 مليار رمز (token) عربي بطول سياق 2048. نموذج أساس (Base) بدون ضبط تعليمات أو محادثة. الهدف: إثبات إمكانية تدريب نموذج عربي مفيد على نطاق صغير جدًا يعمل حتى على المعالج (CPU). المعمارية وسكربتات التدريب مشتقة من SupraLabs/Supra-50M-Base.

Model details


Parameters	~51.8M
Architecture	Llama (`LlamaForCausalLM`)
Hidden size	512 · Layers 12 · Heads 8 (GQA, 4 KV) · head_dim 64
Intermediate size	1408
Vocab size	32000 (custom Arabic byte-level BPE)
Context length	2048
Positional encoding	RoPE (θ = 10000)
Tied embeddings	yes
Type	Base / pretrained (no chat template)
License	Apache-2.0

Training

Corpus: kaust-generative-ai/fineweb-edu-ar (config ar), streamed and tokenized to a memory-mapped token bin.
Budget: 20,000,000,000 tokens, sequence length 2048, 1 epoch over the token stream.
Tokenizer: custom byte-level BPE (ByteLevelBPETokenizer), vocab 32000, specials <s> / </s> / <unk> / <pad>.
Optimizer: AdamW (fused), β = (0.9, 0.95), weight decay 0.1, grad clip 1.0.
Schedule: lr 6e-4, cosine decay, warmup ratio 0.02.
Precision / compute: bf16, torch.compile, effective batch 128 (per-device 16 × grad-accum 8).

The architecture and training loop follow SupraLabs/Supra-50M-Base ("Project Chimera — 50M Llama"). The pretraining script (train.py) is included in this repository.

Usage

The tokenizer uses the TokenizersBackend class, which requires transformers>=5.12. This is a base model — prompt it as a text completer, not a chat assistant:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "oddadmix/50M-2048-Emhotob"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)

prompt = "اللغة العربية هي"
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **ids, max_new_tokens=128, do_sample=True,
    temperature=0.8, top_p=0.9, repetition_penalty=1.2,
)
print(tok.decode(out[0], skip_special_tokens=True))

Fine-tuned variants

Model	What it does
`oddadmix/Nawah-50M-Egyptian-18y-Persona`	Speaks with an 18-year-old Egyptian persona
`oddadmix/Emhotob-50M-GPRO-Arabic-Final`	Arabic tool-calling / function-calling (GRPO-tuned)

Intended use & limitations

Intended use. A from-scratch Arabic base model for research and as a fine-tuning starting point; a CPU-friendly baseline for tiny Arabic SLM experiments.

Limitations. As a base model it does not follow instructions or hold a conversation out of the box — fine-tune it first. At ~50M parameters it is a proof of concept: expect limited world knowledge, weak reasoning, and repetition (use a repetition_penalty). Training data is web text (fineweb-edu-ar), so it carries that corpus's biases. Not suitable for factual, medical, legal, or financial use.

Credits

Architecture & pretraining recipe: SupraLabs/Supra-50M-Base — "Project Chimera" (Apache-2.0).
Pre-training corpus: kaust-generative-ai/fineweb-edu-ar.

Downloads last month: 566

Safetensors

Model size

51.8M params

Tensor type

F32

Model tree for oddadmix/50M-2048-Emhotob

Finetunes

2 models

oddadmix
/

50M-2048-Emhotob