Llama3 8B Chinese Chat ExPO The extrapolated (ExPO) model based on shenzhi wang/Llama3 8B Chinese Chat and meta llama/Meta Llama 3 8B Instruct , as in the "Weak to Strong Extrapolation Expedites Alignment" paper. Specifically, we obtain this model by extrapolating (alpha = 0.3) from the weights of the SFT and DPO/RLHF checkpoints, achieving superior alignment with human preference. Note: This is an experimental model, as I have not comprehensively evaluated its Chinese ability. Unexpected issues may occur when we apply extrapolation to the DPO/RLHF alignment training for new languages (e.g., Chinese). Evaluation Results Evaluation results on the AlpacaEval 2.0 benchmark (you can find the evaluation outputs on the official GitHub repo): Win Rate (Ori) LC Win Rate (Ori) Win Rate (+ ExPO) LC Win Rate (+ ExPO) HuggingFaceH4/zephyr 7b alpha 6.7% 10.0% 10.6% 13.6% HuggingFaceH4/zephyr 7b beta 10.2% 13.2% 11.1% 14.0% berkeley nest/Starling LM 7B alpha 15.0% 18.3% 18.2% 19.5% Nexusflow/Starling LM 7B beta 26.6% 25.8% 29.6% 26.4% snorkelai/Snorkel Mistral PairRM 24.7% 24.0% 28.8% 26.4% RLHFlow/LLaMA3 iterative DPO final 29.2% 36.0% 32.7% 37.8% internlm/internlm2 chat 1.8b 3.8% 4.0% 5.2% 4.3…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy