USER bge m3 U niversal S entence E ncoder for R ussian (USER) is a sentence transformer model for extracting embeddings exclusively for Russian language. It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search. This model is initialized from TatonkaHF/bge m3 en ru which is shrinked version of baai/bge m3 model and trained to work mainly with the Russian language. Its quality on other languages was not evaluated. Usage Using this model becomes easy when you have sentence transformers installed: Then you can use the model like this: However, you can use model directly with transformers Also, you can use native FlagEmbedding library for evaluation. Usage is described in bge m3 model card. Training Details We follow the USER base model training algorithm, with several changes as we use different backbone. Initialization: TatonkaHF/bge m3 en ru – shrinked version of baai/bge m3 to support only Russian and English tokens. Fine tuning: Supervised fine tuning two different models based on data symmetry and then merging via LM Cocktail : 1. Since we split the data, we could additionally apply the AnglE loss to the…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy