DziriBERT DziriBERT is the first Transformer based Language Model that has been pre trained specifically for the Algerian Dialect. It handles Algerian text contents written using both Arabic and Latin characters. It sets new state of the art results on Algerian text classification datasets, even if it has been pre trained on much less data (~1 million tweets). For more information, please visit our paper: https://arxiv.org/pdf/2109.12346.pdf. How to use You can find a fine tuning script in our Github repo: https://github.com/alger ia/dziribert Limitations The pre training data used in this project comes from social media (Twitter). Therefore, the Masked Language Modeling objective may predict offensive words in some situations. Modeling this kind of words may be either an advantage (e.g. when training a hate speech model) or a disadvantage (e.g. when generating answers that are directly sent to the end user). Depending on your downstream task, you may need to filter out such words especially when returning automatically generated text to the end user. How to cite Contact Please contact amine.abdaoui.nlp@gmail.com for any question, feedback or request.
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy