Ettin: an Open Suite of Paired Encoders and Decoders 🎯 TL;DR : State of the art paired encoder and decoder models (17M 1B params) trained identically for fair comparison with open data. Encoders beat ModernBERT. Decoders beat Llama 3.2/SmolLM2. 📄 Paper (Coming Soon) 🚀 GitHub Repository This model is part of the Ettin suite the first collection of paired encoder only and decoder only models trained with identical data, architecture, and training recipes. Ettin enables fair comparisons between encoder and decoder architectures across multiple scales, providing state of the art performance for open data models in their respective size categories. Table of Contents Performance Highlights Quick Start Model Description Training Data Model Family Encoder Models Decoder Models Cross Objective Models Accessing Training Checkpoints Research Applications Training Details Model Architecture Usage Examples Fine tuning Examples Citation 📊 Performance Highlights Encoder Tasks (vs. ModernBERT) GLUE Average : 88.9 vs 88.4 (Base), 90.8 vs 90.4 (Large) MTEB v2 English Retrieval : 45.7 vs 43.9 (Base), 48.4 vs 47.0 (Large) Code Search and Long Context : Superior performance on CodeSearchNet and MLD…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy