LaBSE Model description Language agnostic BERT Sentence Encoder (LaBSE) is a BERT based model trained for sentence embedding for 109 languages. The pre training process combines masked language modeling with translation language modeling. The model is useful for getting multilingual sentence embeddings and for bi text retrieval. Model: HuggingFace's model hub. Paper: arXiv. Original model: TensorFlow Hub. Blog post: Google AI Blog. Conversion from TensorFlow to PyTorch: GitHub. This is migrated from the v2 model on the TF Hub, which uses dict based input. The embeddings produced by both the versions of the model are equivalent. Usage Using the model: To get the sentence embeddings, use the pooler output: Output for other languages: For similarity between sentences, an L2 norm is recommended before calculating the similarity: Details Details about data, training, evaluation and performance metrics are available in the original paper. BibTeX entry and citation info
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy