bert small amharic This model has the same architecture as bert small and was pretrained from scratch using the Amharic subsets of the oscar, mc4, and amharic sentences corpus datasets, on a total of 290 Million tokens. The tokenizer was trained from scratch on the same text corpus, and had a vocabulary size of 28k. It achieves the following results on the evaluation set: Loss: 2.77 Perplexity: 15.96 Even though this model only has 27.8 Million parameters, its performance is comparable to the 10x larger 279 Million parameter xlm roberta base multilingual model on the same Amharic evaluation set. How to use You can use this model directly with a pipeline for masked language modeling: Finetuning This model was finetuned and evaluated on the following Amharic NLP tasks Sentiment Classification Dataset: amharic sentiment Code: https://github.com/rasyosef/amharic sentiment classification Named Entity Recognition Dataset: amharic named entity recognition Code: https://github.com/rasyosef/amharic named entity recognition News Category Classification Dataset: amharic news category classification Code: https://github.com/rasyosef/amharic news category classification Finetuned Model Performa…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy