potion code 16M Model Card Overview This model is superseded by potion code 16M v2. We recommend using that instead for better performance at the same size. potion code 16M is a fast static code embedding model optimized for code retrieval tasks. It powers Semble, a code search library for agents. It is distilled from nomic ai/CodeRankEmbed and trained on the CornStack code corpus using Tokenlearn and contrastive fine tuning. It uses static embeddings, allowing text and code embeddings to be computed orders of magnitude faster than transformer based models on both GPU and CPU. Installation Usage How it works potion code 16M is created using the following pipeline: 1. Vocabulary mining : code specific tokens are mined from CornStack and added to the base CodeRankEmbed tokenizer (42k extra tokens → ~62.5k total) 2. Distillation : the extended vocabulary is distilled from CodeRankEmbed using Model2Vec (256 dimensional embeddings, PCA whitening) 3. Tokenlearn : the distilled model is fine tuned on 240k (query, document) pairs from CornStack using cosine similarity loss 4. Contrastive fine tuning : the model is further fine tuned using MultipleNegativesRankingLoss on 120k CornStack quer…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy