PubMedBERT Embeddings Matryoshka This is a version of PubMedBERT Embeddings with Matryoshka Representation Learning applied. This enables dynamic embeddings sizes of 64 , 128 , 256 , 384 , 512 and the full size of 768 . It's important to note while this method saves space, the same computational resources are used regardless of the dimension size. Sentence Transformers 2.4 added support for Matryoshka Embeddings. More can be read in this blog post. Usage (txtai) This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG). Usage (Sentence Transformers) Alternatively, the model can be loaded with sentence transformers. Usage (Hugging Face Transformers) The model can also be used directly with Transformers. Evaluation Results Performance of this model compared to the top base models on the MTEB leaderboard is shown below. A popular smaller model was also evaluated along with the most downloaded PubMed similarity model on the Hugging Face Hub. The following datasets were used to evaluate model performance. PubMed QA Subset: pqa labeled, Split: train, Pair: (question, long answer) PubMed Subset…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy