gemma 4 31B it GGUF Models Model Generation Details This model was generated using llama.cpp at commit 7c158fbb4 . Quantization Beyond the IMatrix I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides. In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the tensor type option in llama.cpp to manually "bump" important layers to higher precision. You can see the implementation here: 👉 Layer bumping with llama.cpp While this does increase model file size, it significantly improves precision for a given quantization level. I'd love your feedback—have you tried this? How does it perform for you? Click here to get info on choosing the right GGUF model format Hugging Face GitHub Launch Blog Documentation License : Apache 2.0 Authors : Google DeepMind Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on small models) and generating text output. This release includes open weights models…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy