Smaug Llama 3 70B Instruct 32K Built with Meta Llama 3 This is a 32K version of Smaug Llama 3 70B Instruct. It uses PoSE (https://arxiv.org/abs/2309.10400) and LoRA (https://arxiv.org/abs/2106.09685) adapter transfer. More details are coming soon. Needle In A Haystack (https://github.com/jzhang38/EasyContext) heatmap: Model Description Developed by: Abacus.AI License: https://llama.meta.com/llama3/license/ Finetuned from model: meta llama/Meta Llama 3 70B Instruct. How to use The prompt format is unchanged from Llama 3 70B Instruct. Use with transformers See the snippet below for usage with Transformers: Evaluation Arena Hard Arena Hard Score vs selected others (sourced from: (https://lmsys.org/blog/2024 04 19 arena hard/ full leaderboard with gpt 4 turbo as judge)). GPT 4o and Gemini 1.5 pro latest were missing from the original blob post, and we produced those numbers from a local run using the same methodology. Model Score 95% Confidence Interval Average Tokens : : : : GPT 4 Turbo 2024 04 09 82.6 ( 1.8, 1.6) 662 GPT 4o 78.3 ( 2.4, 2.1) 685 Gemini 1.5 pro latest 72.1 ( 2.3, 2.2) 630 Claude 3 Opus 20240229 60.4 ( 3.3, 2.4) 541 Smaug Llama 3 70B Instruct 32K 60.0 ( 2.6, 2.1) 844 Sm…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy