Model Details Model Card for OLMo 2 7B We introduce OLMo 2, a new family of 7B and 13B models featuring a 9 point increase in MMLU, among other evaluation improvements, compared to the original OLMo 7B model. These gains come from training on OLMo mix 1124 and Dolmino mix 1124 datasets and staged training approach. OLMo is a series of O pen L anguage Mo dels designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. Size Training Tokens Layers Hidden Size Attention Heads Context Length OLMo 2 7B 4 Trillion 32 4096 32 4096 OLMo 2 13B 5 Trillion 40 5120 40 4096 The core models released in this batch include the following: Stage OLMo 2 7B OLMo 2 13B Base Model allenai/OLMo 2 1124 7B allenai/OLMo 2 1124 13B SFT allenai/OLMo 2 1124 7B SFT allenai/OLMo 2 1124 13B SFT DPO allenai/OLMo 2 1124 7B DPO allenai/OLMo 2 1124 13B DPO Final Models (RLVR) allenai/OLMo 2 1124 7B Instruct allenai/OLMo 2 1124 13B Instruct Reward Model (RM) allenai/OLMo 2 1124 7B RM (Same as 7B) Installation OLMo 2 will be supported in the next version of Transformers, and you need to inst…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy