Model Details Model Card for OLMo 2 1B We introduce OLMo 2 1B, the smallest model in the OLMo 2 family. OLMo 2 was pre trained on OLMo mix 1124 and uses Dolmino mix 1124 for mid training. OLMo 2 is the latest in a series of O pen L anguage Mo dels designed to enable the science of language models. We have released all code, checkpoints, logs, and associated training details on GitHub. Size Training Tokens Layers Hidden Size Attention Heads Context Length OLMo 2 1B 4 Trillion 16 2048 16 4096 OLMo 2 7B 4 Trillion 32 4096 32 4096 OLMo 2 13B 5 Trillion 40 5120 40 4096 OLMo 2 32B 6 Trillion 64 5120 40 4096 The core models released in this batch include the following: Stage OLMo 2 1B OLMo 2 7B OLMo 2 13B OLMo 2 32B Base Model allenai/OLMo 2 0425 1B allenai/OLMo 2 1124 7B allenai/OLMo 2 1124 13B allenai/OLMo 2 0325 32B SFT allenai/OLMo 2 0425 1B SFT allenai/OLMo 2 1124 7B SFT allenai/OLMo 2 1124 13B SFT allenai/OLMo 2 0325 32B SFT DPO allenai/OLMo 2 0425 1B DPO allenai/OLMo 2 1124 7B DPO allenai/OLMo 2 1124 13B DPO allenai/OLMo 2 0325 32B DPO Final Models (RLVR) allenai/OLMo 2 0425 1B Instruct allenai/OLMo 2 1124 7B Instruct allenai/OLMo 2 1124 13B Instruct allenai/OLMo 2 0325 32B Instruc…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy