HelpSteer2: Open source dataset for training top performing reward models HelpSteer2 is an open source Helpfulness Dataset (CC BY 4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses. This dataset has been created in partnership with Scale AI. When used to tune a Llama 3.1 70B Instruct Model, we achieve 94.1% on RewardBench, which makes it the best Reward Model as of 1 Oct 2024. This reward model is available on HuggingFace in both .nemo format at Llama 3.1 Nemotron 70B Reward or HF compatible format at Llama 3.1 Nemotron 70B Reward HF Using this reward model for RLHF (specifically, REINFORCE), we were able to align a Llama 3.1 70B Instruct model to reach AlpacaEval 2 LC of 57.6, Arena Hard of 85.0 and GPT 4 Turbo MT Bench of 8.98, which are known to be predictive of LMSys Chatbot Arena Elo This Instruct model is available at Llama 3.1 Nemotron 70B Instruct as .nemo model and Llama 3.1 Nemotron 70B Instruct HF as a HF Transformers model. As of 1 Oct 2024, this aligned model is 1 on all three automatic alignment benchmarks, edging out strong frontier models such as…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy