BIA-NLLB-600M-5E-CD

Model Description

BIA-NLLB-600M-5E-CD is a fine-tuned version of Facebook's NLLB (No Language Left Behind) model specifically optimized for Moore language translation. This model enables high-quality translation between Moore and other languages including French and English, supporting the linguistic needs of Burkina Faso and the wider Moore-speaking community.

Our model is built upon the powerful NLLB Transformer architecture, leveraging state-of-the-art multilingual translation techniques. It has been fine-tuned on curated parallel corpora to ensure accurate and contextually appropriate translations for Moore speakers.

Language Information

• Primary Language: Moore (Mooré)
• Language Code: moor_Latn
• Translation Pairs: Moore ↔ French, Moore ↔ English
• Region: Burkina Faso, West Africa
• Speakers: ~5 million native speakers
• Language Family: Niger-Congo → Gur → Oti-Volta → Moore

Model Performance

The model achieves the following results on the evaluation set:

• Initial Validation Loss: 4.7701
• Step 1000 Validation Loss: 1.7923
• Step 2000 Validation Loss: 1.6354
• Step 3000 Validation Loss: 1.5882
• Parameters: 600M
• Training Steps: 3000+ (5 epochs)

Usage

Using our model for translation is simple and straightforward. You can integrate it into your projects via the Hugging Face Transformers library:

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("burkimbia/BIA-NLLB-600M-5E-CD")
model = AutoModelForSeq2SeqLM.from_pretrained("burkimbia/BIA-NLLB-600M-5E-CD")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

def translate(text, src_lang="fra_Latn", tgt_lang="moor_Latn", max_length=128):

    # Tokenize input
    inputs = tokenizer(
        text,
        return_tensors="pt",
        padding=True,
        truncation=True,
        max_length=max_length
    ).to(device)
    
    with torch.no_grad():
        generated_tokens = model.generate(
            **inputs,
            max_length=max_length,
            num_beams=5,  # Beam search for better quality
            early_stopping=True,
            repetition_penalty=1.2,  # Avoid repetitions
            length_penalty=1.0,  # Balance output length
            do_sample=False
        )
    
    translation = tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)[0]
    return translation

# Example usage
french_text = "Le château d'eau est-il loin de votre maison ?"
moore_translation = translate(french_text, src_lang="fra_Latn", tgt_lang="moor_Latn")
print(f"French: {french_text}")
print(f"Moore: {moore_translation}")
# Sato wã zãra ne yãmb yirã bɩ??


# Translate from Moore to French
moore_text = "Ãdũni wã yɛllã yaa wãn to-to n kẽed ne ekollã?
french_translation = translate(moore_text, src_lang="moor_Latn", tgt_lang="fra_Latn")
print(f"Moore: {moore_text}")
print(f"French: {french_translation}")
# Qu'est-ce que la proximité de l'eau apporte à l'école ?

Model Details

Training Configuration

• Base Model: NLLB (No Language Left Behind)
• Model Size: 600M parameters
• Training Steps: 6,436 steps per epoch
• Epochs: 5
• Batch Size: 16
• Gradient Accumulation Steps: 1
• Learning Rate: 1e-4
• Warmup Steps: 1000
• Max Length: 128 tokens
• Hardware: NVIDIA A100-SXM4-40GB (40GB VRAM)
• Framework: PyTorch with FP16 mixed precision
• Optimizer: AdamW with gradient scaling
• Validation Frequency: Every 1000 steps

Training Progress

The model showed consistent improvement during training:

• Rapid initial improvement (4.77 → 1.79 loss in first 1000 steps)
• Continued refinement through subsequent epochs
• Best model checkpoints saved at validation milestones

Limitations

• The model performs best on general domain text (administration, health, daily life, religious)
• Specialized technical may require domain-specific fine-tuning
• Translation quality depends on input text clarity and length
• Continuous improvements are being made with additional training data

Feedback

We're continuously striving to improve our model's performance and usability. If you have any feedback, suggestions, or encounter any issues, please don't hesitate to reach out to us at BurkimbIA.

Citation

If you use this model in your research, please cite:

@misc{bia-nllb-600m,
  title={BIA-NLLB-600M-5E-CD: A Fine-tuned NLLB Model for Moore Language Translation},
  author={ BurkimbIA team},
  year={2025},
  howpublished={\url{https://huggingface.co/burkimbia/BIA-NLLB-600M-5E-CD}}
}

Acknowledgments

• Meta AI for the NLLB architecture and pretrained models
• Moore language community in Burkina Faso
• Translation contributors who provided high-quality parallel data
• BurkimbIA team for data curation and model development

Framework Versions

• Transformers: 4.57.1
• PyTorch: 2.2.1+cu121
• Python: 3.12+

---

Developed by BurkimbIA - Advancing African Language Technologies