gemma 4 31B it AWQ 6Bit Base model: google/gemma 4 31B it This repo quantizes the model using data free quantization (no calibration dataset required). Per layer quantization strategy is applied to achieve, on average, 6 bit quantization level. 【Dependencies / Installation】 As of 2026 04 03 , make sure your system has cuda12.8 installed. Then, create a fresh Python environment (e.g. python3.12 venv) and run: Gemma4 vLLM Official Guide 【vLLM Startup Command】 【Logs】 【Model Files】 File Size Last Updated 24GiB 2026 04 03 【Model Download】 【Overview】 Hugging Face GitHub Launch Blog Documentation License : Apache 2.0 Authors : Google DeepMind Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on small models) and generating text output. This release includes open weights models in both pre trained and instruction tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture of Experts (MoE) architectures, Gemma 4 is well suited for tasks like text generation, coding, and reasoning. The models are avai…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy