xlm mlm en 2048 Table of Contents 1. Model Details 2. Uses 3. Bias, Risks, and Limitations 4. Training 5. Evaluation 6. Environmental Impact 7. Citation 8. Model Card Authors 9. How To Get Started With the Model Model Details The XLM model was proposed in Cross lingual Language Model Pretraining by Guillaume Lample and Alexis Conneau. It’s a transformer pretrained with either a causal language modeling (CLM) objective (next token prediction), a masked language modeling (MLM) objective (BERT like), or a Translation Language Modeling (TLM) object (extension of BERT’s MLM to multiple language inputs). This model is trained with a masked language modeling objective on English text. Model Description Developed by: Researchers affiliated with Facebook AI, see associated paper and GitHub Repo Model type: Language model Language(s) (NLP): English License: CC BY NC 4.0 Related Models: Other XLM models Resources for more information: Cross lingual Language Model Pretraining by Guillaume Lample and Alexis Conneau (2019) Unsupervised Cross lingual Representation Learning at Scale by Conneau et al. (2020) GitHub Repo Hugging Face XLM docs Uses Direct Use The model is a language model. The model…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy