🦣 MAmmoTH2: Scaling Instructions from the Web Project Page: https://tiger ai lab.github.io/MAmmoTH2/ Paper: https://arxiv.org/pdf/2405.03548 Code: https://github.com/TIGER AI Lab/MAmmoTH2 Introduction Introducing 🦣 MAmmoTH2, a game changer in improving the reasoning abilities of large language models (LLMs) through innovative instruction tuning. By efficiently harvesting 10 million instruction response pairs from the pre training web corpus, we've developed MAmmoTH2 models that significantly boost performance on reasoning benchmarks. For instance, MAmmoTH2 7B (Mistral) sees its performance soar from 11% to 36.7% on MATH and from 36% to 68.4% on GSM8K, all without training on any domain specific data. Further training on public instruction tuning datasets yields MAmmoTH2 Plus, setting new standards in reasoning and chatbot benchmarks. Our work presents a cost effective approach to acquiring large scale, high quality instruction data, offering a fresh perspective on enhancing LLM reasoning abilities. Base Model MAmmoTH2 MAmmoTH2 Plus : : : : 7B Mistral 🦣 MAmmoTH2 7B 🦣 MAmmoTH2 7B Plus 8B Llama 3 🦣 MAmmoTH2 8B 🦣 MAmmoTH2 8B Plus 8x7B Mixtral 🦣 MAmmoTH2 8x7B 🦣 MAmmoTH2 8x7B Plu…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy