fastText (Language Identification) fastText is an open source, free, lightweight library that allows users to learn text representations and text classifiers. It works on standard, generic hardware. Models can later be reduced in size to even fit on mobile devices. It was introduced in this paper. The official website can be found here. This LID (Language IDentification) model is used to predict the language of the input text, and the hosted version ( lid218e ) was released as part of the NLLB project and can detect 217 languages. You can find older versions (ones that can identify 157 languages) on the official fastText website. Model description fastText is a library for efficient learning of word representations and sentence classification. fastText is designed to be simple to use for developers, domain experts, and students. It's dedicated to text classification and learning word representations, and was designed to allow for quick model iteration and refinement without specialized hardware. fastText models can be trained on more than a billion words on any multicore CPU in less than a few minutes. It includes pre trained models learned on Wikipedia and in over 157 different la…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy