ModernBERT Cross Encoder: Semantic Similarity (STS) Cross encoders are high performing encoder models that compare two texts and output a 0 1 score. I've found the cross encoders/roberta large stsb model to be very useful in creating evaluators for LLM outputs. They're simple to use, fast and very accurate. Like many people, I was excited about the architecture and training uplift from the ModernBERT architecture ( answerdotai/ModernBERT base ). So I've applied it to the stsb cross encoder, which is a very handy model. Additionally, I've added pretraining from a much larger semi synthetic dataset dleemiller/wiki sim that targets this kind of objective. The inference performance efficiency, expanded context and simplicity make this a really nice platform as an evaluator model. Features High performing: Achieves Pearson: 0.9162 and Spearman: 0.9122 on the STS Benchmark test set. Efficient architecture: Based on the ModernBERT base design (149M parameters), offering faster inference speeds. Extended context length: Processes sequences up to 8192 tokens, great for LLM output evals. Diversified training: Pretrained on dleemiller/wiki sim and fine tuned on sentence transformers/stsb . Pe…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy