⚠️ IMPORTANT NOTICE ⚠️ This is the Dolma 3 Longmino pool ; it hasn't been mixed. If you are interested in the data used to train: Olmo 3 7B: allenai/dolma3 longmino mix 50B 1025 Olmo 3 32B: allenai/dolma3 dolmino mix 100B 1125 Dolma 3 Longmino Pool (639B) Dolma 3 Longmino Pool is the full pool of documents considered for stage 3 (long context) extension trainin of Olmo 3 7B. Dataset Sources Source Type Tokens Docs LC s2pdf REX 32k 64k Synth PDFs 24.1B 492K LC s2pdf CWE 32k 64k Synth PDFs 8.77B 189K LC s2pdf 32k 64k PDFs 106B 2.30M LC s2pdf 8k 32k (8 16k) PDFs 144B 12.7M LC s2pdf 8k 32k (16 32k) PDFs 115B 5.06M LC s2pdf 64k 128k PDFs 96.0B 1.05M LC s2pdf 128k 256k PDFs 60.8B 342K LC s2pdf 256k 512k PDFs 35.1B 97.1K LC s2pdf 512k 1M PDFs 21.5B 30.2K LC s2pdf 1M+ PDFs 26.9B 12.2K Total 639B 22.3M Licensing Information Dolma 3 Longmino is licensed under the Open Data Commons Attribution License v1.0 (ODC By). It is intended for research and educational use. For more information, please see our Responsible Use Guidelines. Citation
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy