""" Public release of the v6 weights as **oddadmix/Nawah-Math-Reasoning**. This is `push_model_v6.py` retargeted at a public repo under the release name. The differences that matter: * `private=False` — this is the release, unlike every rung of the v1→v6 ladder. * The card is written for someone who has never seen the ladder. v1–v5 stay private, so the comparison table names them but does not link them, and the text does not send readers to repos they cannot open. * The training code ships **inside the repo** under `code/`, so the recipe and the weights are one link. `code/README.md` is generated by `build_release_code.py`. Every number in the card is still read from an eval file on disk — nothing is typed by hand: Nawah-Reasoning-v6/eval_reasoning.json v6 on the 1,800-row v6 mix eval (by_source, 4 sources) Nawah-Reasoning-v6/eval_on_synth.json v6 on the same 1,000-row synth set v4/v5 used Nawah-Reasoning-v5/eval_reasoning.json v5 on the 1,400-row v5 mix eval Nawah-Reasoning-v5/eval_on_synth.json v5, same 1,000 synth rows Nawah-Reasoning-v5/eval_on_rel.json v5 on v6's 400 held-out relational rows (the baseline) Nawah-Reasoning-v4/eval_on_mix.json v4 on v3's mix eval Nawah-Reasoning-v4/eval_on_synth.json v4, same 1,000 synth rows Nawah-Reasoning-v3/eval_reasoning.json v3 on the same mix eval Nawah-Reasoning-v3/eval_on_synth.json v3, same 1,000 synth rows Usage: python push_release.py [--repo oddadmix/Nawah-Math-Reasoning] [--dry-run] [--no-code] """ import argparse import json from pathlib import Path from huggingface_hub import HfApi MODEL_DIR = Path("./Nawah-Reasoning-v6") V5_DIR = Path("./Nawah-Reasoning-v5") V4_DIR = Path("./Nawah-Reasoning-v4") V3_DIR = Path("./Nawah-Reasoning-v3") CODE_DIR = Path("./release_code") BASE = "oddadmix/50M-2048-Emhotob" DEMO = "oddadmix/Nawah-Math-Reasoning-Demo" CARD = """--- license: apache-2.0 language: - ar base_model: {base} datasets: - oddadmix/arabic-math-reasoning-synth - oddadmix/gsm8k-reasoning-ar - Omartificial-Intelligence-Space/Arabic_Reasoning_Dataset library_name: transformers pipeline_tag: text-generation tags: - arabic - reasoning - chain-of-thought - math - gsm8k - small-language-model - slm - llama - sft --- # Nawah-Math-Reasoning — نموذج استدلال رياضي عربي A **{params:.1f}M-parameter** Arabic math reasoning model. It writes its derivation step by step inside `…`, then gives the answer. It is small enough to run on a CPU. > **بالعربية:** نموذج عربي صغير (~{params:.0f} مليون معامل) لحل المسائل الحسابية: يكتب خطوات > تفكيره داخل وسم `` ثم يعطي الإجابة. صغير بما يكفي ليعمل على المعالج (CPU). | | | |---|---| | 🤗 **Demo** | [`{demo}`](https://huggingface.co/spaces/{demo}) | | 🧩 **Base model** | [`{base}`](https://huggingface.co/{base}) — Llama architecture, 12 layers, hidden 512, 2048 ctx, pre-trained from scratch on ~20B Arabic tokens | | 📚 **Data** | [`arabic-math-reasoning-synth`](https://huggingface.co/datasets/oddadmix/arabic-math-reasoning-synth) · [`gsm8k-reasoning-ar`](https://huggingface.co/datasets/oddadmix/gsm8k-reasoning-ar) · [`Arabic_Reasoning_Dataset`](https://huggingface.co/datasets/Omartificial-Intelligence-Space/Arabic_Reasoning_Dataset) | | 🛠️ **Training code** | [`code/`](https://huggingface.co/{repo}/tree/main/code) in this repo — data generation, translation, SFT, eval, GRPO | | 🔤 **Vocab** | {vocab} (4 chat/reasoning tokens added to the 32000 base vocab) | ## Results Number agreement, greedy decoding. **Every cell is measured on identical held-out rows.** The `Arabic_Reasoning` and `GSM8K-ar` rows are the eval splits fixed at the start of the project and never re-drawn; the synthetic rows are pinned to the same 1,000 items every earlier version was scored on. The `v3 / v4 / v5` columns are internal development runs, kept here because they are what makes the release number mean something. They are not published — the numbers are, so the ablation is readable without them. | eval set | n | v3 | v4 | v5 | **release** | |---|---:|---:|---:|---:|---:| | GSM8K-ar | {g_n} | {v3_gsm:.1f}% | {v4_gsm:.1f}% | {v5_gsm:.1f}% | **{v6_gsm:.1f}%** | | Arabic_Reasoning | {a_n} | {v3_ar:.1f}% | {v4_ar:.1f}% | **{v5_ar:.1f}%** | {v6_ar:.1f}% | | synthetic math | 1000 | {v3_synth:.1f}% | {v4_synth:.1f}% | {v5_synth:.1f}% | **{v6_synth:.1f}%** | | **synthetic relational** | {r_n} | — | — | {v5_rel:.1f}% | **{v6_rel:.1f}%** | **The relational row is what this release adds.** On problems whose difficulty is the *relation* between quantities (`ضعف`, `نصف`, `أكثر بـ…`) rather than the arithmetic, it scores **{v6_rel:.1f}%** where the previous run scores {v5_rel:.1f}% — a **{rel_gain:+.1f} point** gain and the largest single-cell move anywhere in the development ladder. It did not cost the other distributions: GSM8K-ar is simultaneously the best of the series at **{v6_gsm:.1f}%**, and synthetic math gains {synth_delta:+.1f}. The one regression is `Arabic_Reasoning` at **{ar_delta:+.1f}** against v5 — on {a_n} rows that is close to sampling noise, but it is the second consecutive mix where this column is the give. | detail | GSM8K-ar | Arabic_Reasoning | synth math | synth relational | |---|---:|---:|---:|---:| | final-answer number correct | {g_primary:.1f}% | {a_primary:.1f}% | {s_primary:.1f}% | {r_primary:.1f}% | | all numbers match | {v6_gsm:.1f}% | {v6_ar:.1f}% | {s_nums:.1f}% | {v6_rel:.1f}% | | well-formed `` + answer | {g_well:.1f}% | {a_well:.1f}% | {s_well:.1f}% | {r_well:.1f}% | | mean reasoning length | {g_tokens:.0f} tok | {a_tokens:.0f} tok | {s_tokens:.0f} tok | {r_tokens:.0f} tok | *(the synth-math column here is the 400-row mix cell; the {v6_synth:.1f}% in the table above is the 1,000-row set used for the cross-model comparison.)* Reproduce any cell with `code/eval_reasoning.py` — it is the same script for every model and every row, which is the only reason these are comparable. ### The final checkpoint ships, and eval loss disagrees Loss bottoms at **{best_loss:.4f}** (epoch {best_epoch:.2f}) and rises to **{shipped_loss:.4f}** by epoch {epochs} — yet the epoch-{epochs} weights are the better model. This was measured directly on an earlier run whose corpus contained **no repeated rows**, which rules out memorisation: the minimum-loss checkpoint scored 30.9% where the final scored 35.6%. It happened on four consecutive runs. `train_reasoning.py` therefore takes `LOAD_BEST=0`, and that is not an oversight. ## Training mix {train_n:,} rows, {tok_per_epoch:.1f}M tokens/epoch: | source | rows | tokens/epoch | share | |---|---:|---:|---:| | [`oddadmix/arabic-math-reasoning-synth`](https://huggingface.co/datasets/oddadmix/arabic-math-reasoning-synth) | {synth_rows:,} | {synth_tok:.2f}M | {synth_share:.1f}% | | [`oddadmix/gsm8k-reasoning-ar`](https://huggingface.co/datasets/oddadmix/gsm8k-reasoning-ar) | {gsm_rows:,} | {gsm_tok:.2f}M | {gsm_share:.1f}% | | [`Omartificial-Intelligence-Space/Arabic_Reasoning_Dataset`](https://huggingface.co/datasets/Omartificial-Intelligence-Space/Arabic_Reasoning_Dataset) | {ar_rows:,} (5,536 × 3) | {ar_tok:.2f}M | {ar_share:.1f}% | Of the synthetic corpus's {corpus_rows:,} rows, {rel_rows:,} are **relational** problems generated specifically for this release, after a `pass@k` diagnostic showed the previous model went 0/8 on `ضعف`-style problems and a corpus audit found the relation appears in only 1.34% of rows. The synthetic eval split was **pinned, not re-drawn** when those rows were added: re-shuffling would have moved 1,955 of the 2,000 previously held-out items into train, turning that column into a memorisation score. Full fine-tune from the base (not from the previous version). Loss on the assistant turn only, user prompt masked with `-100`. `Arabic_Reasoning` is ~25× smaller than GSM8K, so it is repeated 3×. | | | |---|---| | epochs | {epochs} ({steps:,} steps) | | effective batch | 64 | | learning rate | 3e-4 cosine, 200 warmup steps | | max length | 768 tokens (mix p100 is {p100} — nothing truncated) | | precision | bf16 | | checkpoint | final (`load_best_model_at_end` disabled — it picks the worse model) | | hardware | 1× RTX A6000, ~{minutes} min | ## Usage ```python from transformers import AutoModelForCausalLM, AutoTokenizer import torch model_id = "{repo}" tok = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).eval() messages = [{{"role": "user", "content": "اشترى خالد 4 دفاتر بسعر 15 جنيهًا للدفتر، ودفع بورقة 100 جنيه. كم المبلغ المتبقي؟"}}] prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) ids = tok(prompt, return_tensors="pt") out = model.generate(**ids, max_new_tokens=384, do_sample=False) print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=False)) ``` Split the parts with `re.match(r"\\s*(.*?)(.*)", completion, re.S)`. Decode with `skip_special_tokens=False` — `` and `` are real tokens in this tokenizer, and stripping them destroys the split. It is **single-turn**: one user message per call. Chat history is out of distribution. **Answer style is not something you can request.** The three corpora disagree — GSM8K rows end in a bare numeral, the other two in an `إذن، …` sentence — and arithmetic word problems look alike in all of them, so the model picks a style per prompt. **Score it on number agreement, not exact string match**, and parse the answer by extracting its numbers. ## Limitations At ~{params:.0f}M parameters this is a **proof of concept**, and the honest headline is the synthetic columns — **{v6_synth:.1f}%** and **{v6_rel:.1f}%** on multi-step problems, well below the {v6_gsm:.1f}% it scores on GSM8K's narrower phrasing. Arithmetic is the dominant failure mode: the reasoning is usually structurally right, one computation step is wrong, and the model then stays faithful to its own bad number. Each corpus brings its own defect. The GSM8K half is machine-translated, its 140,969 rows expanding from only 2,814 question patterns, so that score partly reflects narrow phrasing. The synthetic half is verified for **arithmetic, not for sense** — rows survive where every equation checks out but a step introduces an entity never mentioned, or the answer resolves the reverse of what was asked. The `Arabic_Reasoning` half excludes open-ended expository rows (they have no final answer to place after ``), so expository prompts remain out of distribution. Everything is MSA; the synthetic corpus's region axis sets currency and context, not dialect. The Arabic inherits source artifacts including inconsistent gender agreement. Its reasoning trace is not a faithful account of any internal computation. Do not use it for anything consequential. ## Citation ```bibtex @misc{{nawah_math_reasoning_2026, title = {{Nawah-Math-Reasoning: a 52M-parameter Arabic chain-of-thought math model}}, author = {{Ahmed Wasfy}}, year = {{2026}}, url = {{https://huggingface.co/{repo}}} }} ``` """ def main(): ap = argparse.ArgumentParser() ap.add_argument("--repo", default="oddadmix/Nawah-Math-Reasoning") ap.add_argument("--dry-run", action="store_true") ap.add_argument("--no-code", action="store_true", help="skip the code/ upload") ap.add_argument("--minutes", default="85") args = ap.parse_args() def summary(p): return json.loads(Path(p).read_text(encoding="utf-8"))["summary"] v6 = summary(MODEL_DIR / "eval_reasoning.json") v6_synth = summary(MODEL_DIR / "eval_on_synth.json") v5_mix = summary(V5_DIR / "eval_reasoning.json") v5_synth = summary(V5_DIR / "eval_on_synth.json") v5_rel = summary(V5_DIR / "eval_on_rel.json") v4_mix, v4_synth = summary(V4_DIR / "eval_on_mix.json"), summary(V4_DIR / "eval_on_synth.json") v3_mix, v3_synth = summary(V3_DIR / "eval_reasoning.json"), summary(V3_DIR / "eval_on_synth.json") g, a, s, r = (v6["by_source"]["gsm8k_ar"], v6["by_source"]["arabic_reasoning"], v6["by_source"]["synth_math_ar"], v6["by_source"]["synth_relational_ar"]) tm = json.loads(Path(MODEL_DIR, "train_metrics.json").read_text(encoding="utf-8")) cfg = json.loads(Path(MODEL_DIR, "config.json").read_text(encoding="utf-8")) evals = [h for h in tm["log_history"] if "eval_loss" in h] last = max(h["step"] for h in evals) shipped_loss = next(h["eval_loss"] for h in evals if h["step"] == last) best = min(evals, key=lambda h: h["eval_loss"]) v3_gsm, v4_gsm, v5_gsm = (v3_mix["by_source"]["gsm8k_ar"], v4_mix["by_source"]["gsm8k_ar"], v5_mix["by_source"]["gsm8k_ar"]) v3_ar, v4_ar, v5_ar = (v3_mix["by_source"]["arabic_reasoning"], v4_mix["by_source"]["arabic_reasoning"], v5_mix["by_source"]["arabic_reasoning"]) # measured from data_v6_sft/train.jsonl with the base tokenizer (+12 tok/row of chat scaffold) MIX = {"synth_math_ar": (118062, 16.79, 53.9), "gsm8k_ar": (140969, 11.88, 38.2), "arabic_reasoning": (16608, 2.45, 7.9)} card = CARD.format( repo=args.repo, base=BASE, demo=DEMO, params=51.79, vocab=cfg["vocab_size"], g_n=g["n"], a_n=a["n"], r_n=r["n"], v3_gsm=v3_gsm["numbers_match_pct"], v4_gsm=v4_gsm["numbers_match_pct"], v5_gsm=v5_gsm["numbers_match_pct"], v6_gsm=g["numbers_match_pct"], v3_ar=v3_ar["numbers_match_pct"], v4_ar=v4_ar["numbers_match_pct"], v5_ar=v5_ar["numbers_match_pct"], v6_ar=a["numbers_match_pct"], v3_synth=v3_synth["numbers_match_pct"], v4_synth=v4_synth["numbers_match_pct"], v5_synth=v5_synth["numbers_match_pct"], v6_synth=v6_synth["numbers_match_pct"], v5_rel=v5_rel["numbers_match_pct"], v6_rel=r["numbers_match_pct"], rel_gain=r["numbers_match_pct"] - v5_rel["numbers_match_pct"], ar_delta=a["numbers_match_pct"] - v5_ar["numbers_match_pct"], synth_delta=v6_synth["numbers_match_pct"] - v5_synth["numbers_match_pct"], g_primary=g["primary_number_match_pct"], a_primary=a["primary_number_match_pct"], s_primary=s["primary_number_match_pct"], r_primary=r["primary_number_match_pct"], s_nums=s["numbers_match_pct"], g_well=g["well_formed_pct"], a_well=a["well_formed_pct"], s_well=s["well_formed_pct"], r_well=r["well_formed_pct"], g_tokens=g["mean_reasoning_tokens"], a_tokens=a["mean_reasoning_tokens"], s_tokens=s["mean_reasoning_tokens"], r_tokens=r["mean_reasoning_tokens"], best_loss=best["eval_loss"], best_epoch=best["epoch"], shipped_loss=shipped_loss, epochs=round(max(h.get("epoch", 0) for h in tm["log_history"])), steps=max(h.get("step", 0) for h in tm["log_history"]), train_n=sum(v[0] for v in MIX.values()), tok_per_epoch=sum(v[1] for v in MIX.values()), synth_rows=MIX["synth_math_ar"][0], synth_tok=MIX["synth_math_ar"][1], synth_share=MIX["synth_math_ar"][2], gsm_rows=MIX["gsm8k_ar"][0], gsm_tok=MIX["gsm8k_ar"][1], gsm_share=MIX["gsm8k_ar"][2], ar_rows=MIX["arabic_reasoning"][0], ar_tok=MIX["arabic_reasoning"][1], ar_share=MIX["arabic_reasoning"][2], corpus_rows=120462, rel_rows=20139, p100=703, minutes=args.minutes, ) out = Path("release_README.md") out.write_text(card, encoding="utf-8") print(f"[+] wrote {out} ({len(card)} chars)") if args.dry_run: print("[dry-run] not pushing") return api = HfApi() api.create_repo(args.repo, private=False, exist_ok=True) # Weights first, then the card, so the repo is never public-and-uncarded for long. api.upload_folder( folder_path=str(MODEL_DIR), repo_id=args.repo, # eval_on_synth.json ships: it is the 1,000-row cross-model synth cell the card quotes. # eval_on_v6mix.json does not — it is byte-identical to eval_reasoning.json. ignore_patterns=["checkpoint-*/*", "checkpoint-*", "eval_on_v6mix.json", "README.md"], commit_message="Nawah-Math-Reasoning: weights, tokenizer, eval + training metrics") api.upload_file(path_or_fileobj=str(out), path_in_repo="README.md", repo_id=args.repo, commit_message="model card") if not args.no_code: if not CODE_DIR.is_dir(): raise SystemExit(f"{CODE_DIR} missing — run build_release_code.py first") api.upload_folder(folder_path=str(CODE_DIR), repo_id=args.repo, path_in_repo="code", commit_message="training code: data generation, SFT, eval, GRPO") print(f"[+] https://huggingface.co/{args.repo}") if __name__ == "__main__": main()