Gemma 4 QAT MTP Assistant Heads — Atomic GGUF (Q8 0) Three draft heads for speculative decoding with the official Gemma 4 QAT Q4 0 models. Converted from Google's published unquantized QAT assistant checkpoints into the GGUF format used by the Atomic TurboQuant llama.cpp fork. Using QAT matched heads instead of the generic non QAT heads raised acceptance rates from as low as 56.9% up to 91.8% on the 26B A4B model. Compatibility: These files use the gemma4 assistant architecture shape. They load on: The Atomic TurboQuant llama.cpp fork Stock llama.cpp once PR 23398 ("llama: add Gemma4 MTP") merges — it uses the same architecture name and tensor layout They will not load on stock llama.cpp before that PR, and are not compatible with the ik llama format used by ji farthing/gemma 4 qat q4 0 MTP assistants ik llama GGUF. PARALLEL=2 note: The same n tokens reshape bug that caused 2 slot crashes on the Atomic fork is also present in PR 23398 at the time of writing. Until that is patched upstream, use n parallel 1 regardless of which build you're on. Background — What Is a Draft Head and Why Does Matching Matter? Speculative decoding (called MTP — Multi Token Prediction — in llama.cpp) is…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy