GLuCoSE (General Luke based Contrastive Sentence Embedding) base Japanese 日本語のREADME/Japanese README GLuCoSE (General LUke based COntrastive Sentence Embedding, "glucose") is a Japanese text embedding model based on LUKE. In order to create a general purpose, user friendly Japanese text embedding model, GLuCoSE has been trained on a mix of web data and various datasets associated with natural language inference and search. This model is not only suitable for sentence vector similarity tasks but also for semantic search tasks. Maximum token count: 512 Output dimension: 768 Pooling: mean pooling Supported language: Japanese Usage You can use this model easily with sentence transformers. First, install sentence transformers with pip as follows: You can load the model and convert sentences into dense vectors as shown below: Since the loss function used during training is cosine similarity, we recommend using cosine similarity for downstream tasks. This text embedding model can also be used in LangChain. Please refer to this page for more information. Resources Used The following resources were used to train this model. Pre trained model studio ousia/luke japanese base lite Datasets mC4…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy