MMLW roberta base MMLW (muszę mieć lepszą wiadomość) are neural text encoders for Polish. This is a distilled model that can be used to generate embeddings applicable to many tasks such as semantic similarity, clustering, information retrieval. The model can also serve as a base for further fine tuning. It transforms texts to 768 dimensional vectors. The model was initialized with Polish RoBERTa checkpoint, and then trained with multilingual knowledge distillation method on a diverse corpus of 60 million Polish English text pairs. We utilised English FlagEmbeddings (BGE) as teacher models for distillation. Usage (Sentence Transformers) ⚠️ Our embedding models require the use of specific prefixes and suffixes when encoding texts. For this model, each query should be preceded by the prefix "zapytanie: " ⚠️ You can use the model like this with sentence transformers: Evaluation Results The model achieves an Average Score of 61.05 on the Polish Massive Text Embedding Benchmark (MTEB). See MTEB Leaderboard for detailed results. The model achieves NDCG@10 of 53.60 on the Polish Information Retrieval Benchmark. See PIRB Leaderboard for detailed results. Acknowledgements This model was trai…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy