GTE ModernColBERT v1 PyLate model based on Alibaba NLP/gte modernbert base This is a PyLate model trained on the ms marco en bge gemma dataset. It maps sentences & paragraphs to sequences of 128 dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator. Model Details Model Description Model Type: PyLate model Base model: Alibaba NLP/gte modernbert base Document Length: 300 tokens Query Length: 32 tokens Output Dimensionality: 128 dimensions Similarity Function: MaxSim Training Dataset: ms marco en bge gemma Language: English License: Apache 2.0 Document length GTE ModernColBERT has been trained with knowledge distillation on MS MARCO with a document length of 300 tokens, explaining its default value for documents length. However, as illustrated in the ModernBERT paper, ColBERT models can generalize to documents lengths way beyond their training length and GTE ModernColBERT actually yields results way above SOTA in long context embedding benchmarks, see LongEmbed results. Simply change adapt the document length parameter to your needs when loading the model: ModernBERT itself has only been trained on 8K context length, but it seems that GTE…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy