Introduction We present NV Embed v2, a generalist embedding model that ranks No. 1 on the Massive Text Embedding Benchmark (MTEB benchmark)(as of Aug 30, 2024) with a score of 72.31 across 56 text embedding tasks. It also holds the No. 1 in the retrieval sub category (a score of 62.65 across 15 tasks) in the leaderboard, which is essential to the development of RAG technology. NV Embed v2 presents several new designs, including having the LLM attend to latent vectors for better pooled embedding output, and demonstrating a two staged instruction tuning method to enhance the accuracy of both retrieval and non retrieval tasks. Additionally, NV Embed v2 incorporates a novel hard negative mining methods that take into account the positive relevance score for better false negatives removal. For more technical details, refer to our paper: NV Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. Model Details Base Decoder only LLM: Mistral 7B v0.1 Pooling Type: Latent Attention Embedding Dimension: 4096 How to use Here is an example of how to encode queries and passages using Huggingface transformer and Sentence transformer. Please find the required package version h…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy