Ik45/opus-mt-ar-id-hybrid

OPUS-MT Arabic-to-Indonesian Hybrid Translation Model

Model ID: Ik45/opus-mt-ar-id-hybrid
Architecture: MarianMT (Encoder-Decoder Transformer)
Source Language: Arabic (ar)
Target Language: Indonesian (id)
Model Type: Hybrid Cross-Lingual Transfer Model

---

🎯 Overview

This model is a hybrid neural machine translation model that translates from Arabic to Indonesian, created using a novel encoder-decoder swap technique. Instead of training from scratch, the model leverages pre-trained components from existing OPUS-MT models through strategic component transfer, followed by fine-tuning on parallel Arabic-Indonesian data.

Hybrid Architecture Strategy

Component	Source Model	Description
Encoder	Helsinki-NLP/opus-mt-ar-en	Arabic-to-English encoder, pre-trained to understand Arabic language patterns
Decoder	Helsinki-NLP/opus-mt-en-id	English-to-Indonesian decoder, pre-trained to generate Indonesian text
Shared Embeddings	From Decoder Model	Indonesian vocabulary embeddings preserved from the en-id model

Key Insight: MarianMT uses shared embeddings between encoder and decoder. The decoder determines the output vocabulary, so the decoder model serves as the base while only the encoder layers are replaced.

---

🔧 Technical Details

Architecture Compatibility

The following architecture checks were performed to ensure compatibility between source models:

Check	opus-mt-ar-en	opus-mt-en-id	Status
d_model	512	512	✅ Pass
encoder_layers	6	6	✅ Pass
decoder_layers	6	6	✅ Pass
encoder_attention_heads	8	8	✅ Pass
decoder_attention_heads	8	8	✅ Pass
encoder_ffn_dim	2048	2048	✅ Pass
decoder_ffn_dim	2048	2048	✅ Pass

Vocabulary Size: 54,796 tokens (from Indonesian target vocabulary)
Hidden Dimensions: 512
Attention Heads: 8
Encoder/Decoder Layers: 6 each

Training Configuration

# Fine-tuning Hyperparameters
learning_rate: 3e-5
batch_size: 16
num_train_epochs: 5
max_length: 128
weight_decay: 0.01
fp16: True
optimizer: AdamW

---

📊 Training Data

The model was fine-tuned on a combined corpus of Arabic-Indonesian parallel texts extracted from the OPUS collection:

Dataset Source Sentence Pairs Bible UEDIN Religious texts 31,000 Tanzil Quran translations 77,000 NeuLab-TedTalks TED Talks 4,000 TED2020 TED Talks 2020 8,000 QED Educational dialogues 2,000 Total 536,463

Data preprocessing included:

• Alignment of .ar and .id parallel files
• Minimum length filtering (removing empty lines)
• UTF-8 normalization for Arabic script

---

🚀 Usage

Quick Start

from transformers import MarianMTModel, MarianTokenizer

# Load model and tokenizer
model_name = "Ik45/opus-mt-ar-id-hybrid"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)

# Translate Arabic to Indonesian
def translate(text: str) -> str:
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
    translated = model.generate(
        **inputs,
        max_length=128,
        num_beams=4,
        early_stopping=True
    )
    return tokenizer.decode(translated[0], skip_special_tokens=True)

# Example
arabic_text = "صباح الخير، كيف حالك؟"
indonesian_translation = translate(arabic_text)
print(f"AR: {arabic_text}")
print(f"ID: {indonesian_translation}")

Advanced Usage with Pipeline

from transformers import pipeline

translator = pipeline("translation", model="Ik45/opus-mt-ar-id-hybrid")
result = translator("هذه جملة مثال باللغة العربية.")
print(result[0]['translation_text'])

---

🧪 Model Verification

Architecture Validation

The following tests were performed to verify model integrity:

Test Status Details Config Validity ✅ Pass pad_token_id (54795) < vocab_size (54796) Embedding Shape ✅ Pass torch.Size([54796, 512]) Encoder Forward ✅ Pass Output shape: (batch, seq_len, 512) Decoder Forward ✅ Pass Output shape: (batch, seq_len, 512) Full Generation ✅ Pass Successful token generation Parameter Count ✅ Pass 72,718,336 parameters

Comparison with Parent Models

Model Parameters Vocab Size Pad Token ID opus-mt-ar-en 76,833,792 62,834 62,833 opus-mt-en-id 72,718,336 54,796 54,795 opus-mt-ar-id-hybrid 72,718,336 54,796 54,795

---

⚠️ Limitations

1. Fine-tuning Required: The hybrid architecture requires fine-tuning to align the Arabic encoder with the Indonesian decoder. Without fine-tuning, output quality is very low.

2. Domain Coverage: Training data is primarily from religious texts (Bible, Quran) and TED Talks. Performance may vary on:

   • Technical/scientific texts
   • Classical Arabic (fus'ha)
   • Colloquial Indonesian
   • Legal or medical documents

3. Arabic Input: The model accepts standard Arabic script. It does not support:

   • Transliterated Arabic (Latin script)
   • Arabic dialects (Egyptian, Levantine, Gulf, etc.)
   • Mixed Arabic-English code-switching

4. Sentence Length: Optimal performance for sentences under 50 tokens. Very long sentences may suffer from attention degradation.

5. Training Instability: During initial training attempts, CUDA device-side assert errors were encountered due to vocabulary mismatch issues. The final model uses the correct hybrid tokenizer configuration to avoid index out-of-bounds errors.

---

🔬 Research Notes

Why This Approach Works

The encoder-decoder swap strategy exploits the modular nature of transformer architectures:

1. Encoder Independence: The encoder processes source language into language-agnostic hidden representations
2. Decoder Specialization: The decoder generates target language from these representations
3. Shared Embedding Constraint: MarianMT ties encoder/decoder embeddings, so the decoder's vocabulary must be preserved

By combining an Arabic encoder (trained to map Arabic → hidden space) with an Indonesian decoder (trained to map hidden space → Indonesian), we create a bridge that only needs alignment through fine-tuning.

Comparison with Sister Model

This model is the reverse direction of Ik45/opus-mt-id-ar-hybrid:

`opus-mt-id-ar-hybrid`	`opus-mt-ar-id-hybrid`	

Direction Indonesian → Arabic Arabic → Indonesian Encoder Source opus-mt-id-en opus-mt-ar-en Decoder Source opus-mt-en-ar opus-mt-en-id Vocab Size 62,802 54,796 Parameters 76,817,408 72,718,336

---

📚 Citation

If you use this model in your research, please cite:

@misc{opus-mt-ar-id-hybrid,
  title={OPUS-MT Arabic-to-Indonesian Hybrid Translation Model},
  author={Ik45},
  year={2025},
  howpublished={\url{https://huggingface.co/Ik45/opus-mt-ar-id-hybrid}},
  note={Hybrid model combining encoder from opus-mt-ar-en and decoder from opus-mt-en-id}
}

Parent Models

@article{tiedemann2020opus,
  title={OPUS-MT – Building open translation services for the World},
  author={Tiedemann, Jörg and Thottingal, Santhosh},
  journal={Proceedings of the 22nd Annual Conference of the European Association for Machine Translation},
  year={2020}
}

---

🤝 Acknowledgments

• Helsinki-NLP for the original OPUS-MT models
• OPUS Project for the parallel corpora
• Hugging Face for the Transformers library and model hosting infrastructure

---

📄 License

This model is released under the Apache 2.0 License, following the licensing of the parent OPUS-MT models.

---

🐛 Issues and Contributions

For issues, questions, or contributions related to this model, please open an issue on the Hugging Face model repository or contact the model author.

Model Repository: https://huggingface.co/Ik45/opus-mt-ar-id-hybrid

---

🔗 Related Models

• Reverse Direction: Ik45/opus-mt-id-ar-hybrid — Indonesian to Arabic
• Parent Encoder: Helsinki-NLP/opus-mt-ar-en — Arabic to English
• Parent Decoder: Helsinki-NLP/opus-mt-en-id — English to Indonesian

---