LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders LLM2Vec is a simple recipe to convert decoder only LLMs into text encoders. It consists of 3 simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. The model can be further fine tuned to achieve state of the art performance. Repository: https://github.com/McGill NLP/llm2vec Paper: https://arxiv.org/abs/2404.05961 Installation Usage Questions If you have any question about the code, feel free to email Parishad ( parishad.behnamghader@mila.quebec ) and Vaibhav ( vaibhav.adlakha@mila.quebec ).
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy