Model Details Model Card for OLMo 2 13B We introduce OLMo 2, a new family of 7B and 13B models trained on up to 5T tokens. These models are on par with or better than equivalently sized fully open models, and competitive with open weight models from Meta and Mistral on English academic benchmarks. OLMo is a series of O pen L anguage Mo dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The core models released in this batch include the following: Size Training Tokens Layers Hidden Size Attention Heads Context Length OLMo 2 7B 4 Trillion 32 4096 32 4096 OLMo 2 13B 5 Trillion 40 5120 40 4096 The core models released in this batch include the following: Stage OLMo 2 7B OLMo 2 13B Base Model allenai/OLMo 2 1124 7B allenai/OLMo 2 1124 13B SFT allenai/OLMo 2 1124 7B SFT allenai/OLMo 2 1124 13B SFT DPO allenai/OLMo 2 1124 7B DPO allenai/OLMo 2 1124 13B DPO Final Models (RLVR) allenai/OLMo 2 1124 7B Instruct allenai/OLMo 2 1124 13B Instruct Reward Model (RM) allenai/OLMo 2 1124 7B RM (Same as 7B) Installation OLMo 2 will be supported in the next v…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy