AxionML Gemma 4 12B NVFP4 Developed by AxionML for open source serving and deployment use cases. Part of AxionML's effort to provide ready to serve quantized models for the community. This is an NVFP4 quantized version of google/gemma 4 12B it (11.95B params). It follows NVIDIA's own dense Gemma 4 NVFP4 recipe ( nvidia/Gemma 4 31B IT NVFP4 ): the MLP / feed forward linear layers are quantized to NVFP4, while attention is kept in BF16 . Gemma's attention activations carry large per channel outliers that 4 bit activation quantization cannot represent — so, exactly as NVIDIA does for Gemma 4, only the FFN is taken to FP4. The result is lossless on GSM8K while shrinking the model from ~24 GB (BF16) to ~11 GB. Quantization Details This model was quantized by applying NVFP4 to the weights and activations of the MLP (feed forward) linear operators within the transformer blocks. Attention (q/k/v/o) is kept in BF16. The KV cache is quantized to FP8 (E4M3) . Embeddings, lm head , and the multimodal (vision/audio) embedders are kept in their original BF16 precision. Quantization format NVFP4 — MLP only (W4A4 on FFN, attention BF16), MSE weight calibration Weight micro block / group size 16 (F…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy