DOLMino dataset mix for OLMo2 stage 2 annealing training. Mixture of high quality data used for the second stage of OLMo2 training. Source Sizes Name Category Tokens Bytes (uncompressed) Documents License DCLM HQ Web Pages 752B 4.56TB 606M CC BY 4.0 Flan HQ Web Pages 17.0B 98.2GB 57.3M ODC BY Pes2o STEM Papers 58.6B 413GB 38.8M ODC BY Wiki Encyclopedic 3.7B 16.2GB 6.17M ODC BY StackExchange CodeText 1.26B 7.72GB 2.48M CC BY SA {2.5, 3.0, 4.0} TuluMath Synth Math 230M 1.03GB 220K ODC BY DolminoSynthMath Synth Math 28.7M 163MB 725K ODC BY TinyGSM MIND Synth Math 6.48B 25.52GB 17M ODC BY MathCoder2 Synth Math 3.87B 18.48GB 2.83M Apache 2.0 Metamath owmfilter Math 84.2M 741MB 383K CC BY SA 4.0 CodeSearchNet owmfilter Math 1.78M 29.8MB 7.27K ODC BY GSM8K Math 2.74M 25.3MB 17.6K MIT Total 843B 5.14TB 732M ODC BY Where the breakdowns of each of TuluMath and DolminoSythMath are as follows: Name Category Tokens Bytes (uncompressed) Documents License Personahub math v5 TuluMath 191M 825MB 150K ODC BY Personahub math interm TuluMath 19.7M 82.9MB 20k ODC BY Personahub math grade TuluMath 21.8M 119.7MB 50K ODC BY BasicMathMJ DolminoSynthMath 11.1M 84.7MB 664K ODC BY GSM8K synth DolminoSynthMath…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy