Gemma 4 26B A4B it NVFP4A16 First community NVFP4 quantization of google/gemma 4 26B A4B it — the Mixture of Experts variant of Gemma 4 with 25.2B total parameters and only 3.8B active per token. W4A16 — weights in FP4, activations in FP16 (weight only quantization). Key Specs Original (BF16) NVFP4A16 (this) Size on disk ~49 GB ~16.5 GB Compression — 3.0x Total parameters 25.2B 25.2B Active parameters 3.8B 3.8B Architecture MoE: 128 experts, 8 active/token same Context window 256K tokens 256K tokens Modalities Text, Image, Video Text, Image, Video (all verified) Quantization — W4A16 (FP4 weights, FP16 activations) Serving with vLLM Requirements vLLM build with transformers = 5.4 (for Gemma 4 architecture support) On DGX Spark / SM 12.1: spark vllm docker built with tf5 flag Included gemma4 patched.py for NVFP4 MoE scale key loading (see vLLM Patch) Quick Start Key Flags Flag Why quantization modelopt modelopt NVFP4 checkpoint format moe backend marlin Marlin kernel for MoE expert layers kv cache dtype fp8 Saves memory for longer contexts e VLLM NVFP4 GEMM BACKEND=marlin Marlin for non MoE layers (needed on SM 12.1) trust remote code Required for Gemma 4 Testing This is an instruct…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy