Faro Yi 9B DPO This is the DPO version of wenbopan/Faro Yi 9B. Compared to Faro Yi 9B and Yi 9B 200K, the DPO model excels at many tasks, surpassing the original Yi 9B 200K by a large margin. On the Open LLM Leaderboard, it ranks 2 among all 9B models, 1 among all Yi 9B variants. Metric MMLU GSM8K hellaswag truthfulqa ai2 arc winogrande CMMLU Yi 9B 200K 65.73 50.49 56.72 33.80 69.25 71.67 71.97 Faro Yi 9B 68.80 63.08 57.28 40.86 72.58 71.11 73.28 Faro Yi 9B DPO 69.98 66.11 59.04 48.01 75.68 73.40 75.23 Faro Yi 9B DPO's responses are also favored by GPT 4 Judge in MT Bench How to Use Faro Yi 9B DPO uses the chatml template and performs well in both short and long contexts. For longer inputs under 24GB of VRAM , I recommend to use vLLM to have a max prompt of 32K. Setting kv cache dtype="fp8 e5m2" allows for 48K input length. 4bit AWQ quantization on top of that can boost input length to 160K, albeit with some performance impact. Adjust max model len arg in vLLM or config.json to avoid OOM. Or With Transformers
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy