Naqta (Arabic: نقطة, "dot/period") is a high-performance Arabic punctuation restoration model. Given plain unpunctuated Arabic text, it predicts the correct punctuation mark after each word using token-level sequence classification on top of XLM-RoBERTa Large.

💡 Try it live on the Hugging Face Space

✨ What Does It Restore?

🏆 Results

Validation Metrics (v11d — Final)

Per-Class F1 Score

🗂️ Training Data

The model was trained on a large multi-source Arabic corpus totaling over 1.4 million paragraphs from six diverse sources, covering a broad range of Arabic writing styles and domains.

Corpus Sources

All paragraphs were filtered to contain at least one Arabic letter and one target punctuation mark. Non-target punctuation (e.g., «», …, parentheses) was removed before training.

Punctuation Coverage (raw corpus)

Data Balance Strategy

To prevent the model from ignoring rare punctuation marks, a targeted oversampling strategy was applied:

After oversampling, the combined training pool grew to ~2.4 million paragraphs.

Dataset Splits

• Sliding-window context (window=3 sentences, stride=2) was applied to training data only
• Validation and test sets remain un-windowed for clean, unbiased evaluation
• Splits were stratified by the rarest punctuation mark in each sequence

Preprocessing

• Arabic normalization: alef variants → ا, ya variants → ي, diacritics stripped
• Label assigned per word = punctuation mark following that word
• Multi-subword words: only the first subword receives the label; others are masked (-100)

⚙️ Model Architecture & Training

Two-Phase Training

Class Weights

Rare class weights were additionally boosted:

🚀 Quick Start

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

repo_id = "MostafaMaroof/Naqta"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForTokenClassification.from_pretrained(repo_id)
model.eval()

id2label = model.config.id2label

text = "بلغت نسبة النمو الاقتصادي 4.7 بالمئة خلال الربع الثالث من عام 2024 وهو اعلى مستوى منذ خمس سنوات"
words = text.split()

inputs = tokenizer(
    words,
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=384,
)

with torch.no_grad():
    logits = model(**inputs).logits

pred_ids = logits.argmax(dim=-1)[0].tolist()
word_ids = inputs.word_ids(batch_index=0)

restored_words = []
previous_word_id = None
for token_id, word_id in zip(pred_ids, word_ids):
    if word_id is None or word_id == previous_word_id:
        continue
    word = words[word_id]
    label = id2label[token_id]
    if label != "O":
        word = word + label
    restored_words.append(word)
    previous_word_id = word_id

restored_text = " ".join(restored_words)
print(restored_text)
# → بلغت نسبة النمو الاقتصادي 4.7 بالمئة خلال الربع الثالث من عام 2024، وهو اعلى مستوى منذ خمس سنوات.

📖 Example

Input (unpunctuated):

اذا اردت ان تنجح في حياتك فعليك ان تحدد اهدافك واضحة وان تعمل بجد واستمرارية ولا تيأس عند اول عقبة تواجهها

Output (restored):

اذا اردت ان تنجح في حياتك، فعليك ان تحدد اهدافك واضحة، وان تعمل بجد واستمرارية، ولا تيأس عند اول عقبة تواجهها.

Question example:

من اخترع الهاتف وفي اي سنة تم ذلك وما هي اهمية هذا الاختراع

من اخترع الهاتف، وفي اي سنة تم ذلك، وما هي اهمية هذا الاختراع؟

🎯 Intended Use

Naqta is well-suited for:

• 🎙️ ASR post-processing — restoring punctuation in Arabic speech transcripts
• 📄 Readability enhancement — making raw Arabic text easier to read
• 🔧 NLP preprocessing — improving text quality for downstream Arabic NLP tasks
• 🔬 Research — Arabic punctuation restoration benchmark evaluation

⚠️ Limitations

• Punctuation restoration is partly stylistic — multiple valid outputs may exist for a single input.
• Performance may degrade on highly dialectal, technical, or domain-specific text.
• The model does not predict quotation marks or dialogue markers («»).
• Very short or fragmented text (< 5 words) may produce less reliable results.
• The model predicts punctuation position only and does not perform grammar correction.

📜 License

This model is released under the MIT License.

🔗 Citation

If you use Naqta in your work, please reference:

@misc{naqta2025,
  title     = {Naqta: Arabic Punctuation Restoration with XLM-RoBERTa},
  author    = {MostafaMaroof},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/MostafaMaroof/Naqta}
}