Meta Llama 3 8B Instruct llamafile This repository contains executable weights (which we call llamafiles) that run on Linux, MacOS, Windows, FreeBSD, OpenBSD, and NetBSD for AMD64 and ARM64. Model creator: Meta Original model: meta llama/Meta Llama 3 8B Instruct Quickstart Running the following on a desktop OS will launch a tab in your web browser with a chatbot interface. For further information, please see the llamafile README. Having trouble? See the "Gotchas" section of the README. Prompting Prompt template: History template: About llamafile llamafile is a new format introduced by Mozilla Ocho on Nov 20th 2023. It uses Cosmopolitan Libc to turn LLM weights into runnable llama.cpp binaries that run on the stock installs of six OSes for both ARM64 and AMD64. About Quantization Formats Your choice of quantization format depends on three things: 1. Will it fit in RAM or VRAM? 2. Is your use case reading (e.g. summarization) or writing (e.g. chatbot)? 3. llamafiles bigger than 4.30 GB are hard to run on Windows (see gotchas) Good quants for writing (eval speed) are Q5\ K\ M, and Q4\ 0. Text generation is bounded by memory speed, so smaller quants help, but they also cause the LLM to…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy