Fine Tashkeel: Finetuning Byte Level Models for Accurate Arabic Text Diacritization Table of Contents Introduction Models ByT5 Model Description Benchmarks Citation Contact Introduction Most of previous work on learning diacritization of the Arabic language relied on training models from scratch. In this paper, we investigate how to leverage pre trained language models to learn diacritization. We finetune token free pre trained multilingual models (ByT5) to learn to predict and insert missing diacritics in Arabic text, a complex task that requires understanding the sentence semantics and the morphological structure of the tokens. We show that we can achieve state of the art on the diacritization task with minimal amount of training and no feature engineering, reducing WER by 40%. We release our finetuned models for the greater benefit of the researchers in the community. Model Description The ByT5 model, distinguished by its innovative token free architecture, directly processes raw text to adeptly navigate diverse languages and linguistic nuances. Pre trained on a comprehensive text corpus mc4, ByT5 excels in understanding and generating text, making it versatile for various NLP t…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy