Smaug Llama 3 70B Instruct Built with Meta Llama 3 This model was built using a new Smaug recipe for improving performance on real world multi turn conversations applied to meta llama/Meta Llama 3 70B Instruct. The model outperforms Llama 3 70B Instruct substantially, and is on par with GPT 4 Turbo, on MT Bench (see below). EDIT: Smaug Llama 3 70B Instruct is the top open source model on Arena Hard currently! It is also nearly on par with Claude Opus see below. We are conducting additional benchmark evaluations and will add those when available. Model Description Developed by: Abacus.AI License: https://llama.meta.com/llama3/license/ Finetuned from model: meta llama/Meta Llama 3 70B Instruct. How to use The prompt format is unchanged from Llama 3 70B Instruct. Use with transformers See the snippet below for usage with Transformers: Evaluation Arena Hard Score vs selected others (sourced from: (https://lmsys.org/blog/2024 04 19 arena hard/ full leaderboard with gpt 4 turbo as judge)). GPT 4o and Gemini 1.5 pro latest were missing from the original blob post, and we produced those numbers from a local run using the same methodology. Model Score 95% Confidence Interval Average Tokens…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy