Gemma 4 E4B it NVFP4A16 By coolthor — writing about LLM infrastructure, quantization, and local AI deployment on DGX Spark. First NVFP4 quantization of Google's Gemma 4 E4B it model. NVFP4A16 (4 bit weights, 16 bit activations) quantized using llm compressor from the official BF16 checkpoint google/gemma 4 E4B it. Benchmark — NVIDIA DGX Spark (GB10) Tested on DGX Spark (GB10, 128GB unified memory, 273 GB/s bandwidth): Format GPU Memory tok/s Relative BF16 15.0 GB 19.2 1.0x FP8 (online) 11.4 GB 36.0 1.9x NVFP4A16 9.8 GB 49.9 2.6x NVFP4 W4A4 10.2 GB 39.2 2.0x 3 runs x 500 tokens, +/ 0.1 tok/s variance Long output (1000 tokens): 49.8 tok/s, no degradation Concurrent (3 parallel): 52.7 tok/s per request, 158 tok/s aggregate No repetition degradation observed Why W4A16 beats W4A4 on GB10 GB10 (SM121) lacks native FP4 activation compute. W4A4 falls back to Marlin dequantization for activations, adding overhead that negates the bandwidth savings. On GPUs with native FP4 compute (B200/B100), W4A4 should be faster. On GB10, W4A16 is the optimal format. Known Performance Limitation: Triton Attention Fallback This affects ALL E4B configurations on vLLM (BF16, FP8, NVFP4), not just this checkp…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy