Gemma 4 12B it MXFP4 GGUF MXFP4 (OCP microscaling FP4) quantization of Google's Gemma 4 12B It, a multimodal language model with native vision understanding. This repository contains two files: gemma 4 12b it mxfp4.gguf — Text backbone (48 transformer layers, 3840 hidden dim, 262k context) quantized to MXFP4 mmproj gemma 4 12b it f16.gguf — SigLIP vision encoder + projector at F16 precision (required for image input) About MXFP4 MXFP4 (Microscaling FP4) is the open standard 4 bit floating point format defined by the OCP Microscaling Formats (MX) specification. It uses an E2M1 format (1 sign, 2 exponent, 1 mantissa bit) with per block shared exponent (scaling factor), providing better dynamic range than INT4 while remaining hardware agnostic. Feature MXFP4 Q4 K M NVFP4 Numeric format E2M1 microscaling FP4 INT4 block quantization E4M3 native FP4 Shared exponent (scale) Per 32 elements Per 32 elements None (native FP4) Effective BPW 4.61 ~4.50 4.68 Hardware support Any GPU + CPU Any GPU + CPU Blackwell only Dynamic range (max normal) 30 (E2M1) 7 (INT4, symmetric) 448 (E4M3) Dequantization overhead Moderate (scale mul) Moderate (scale mul) None When to use MXFP4 : You want a universal…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy