DiffusionGemma 26B A4B it — INT8 W8A8 (dynamic) INT8 weight + activation quantization of google/diffusiongemma 26B A4B it, Google's block diffusion language model (dLLM) built on the Gemma 4 26B A4B MoE backbone. Why INT8 instead of FP8? The existing FP8 dynamic quantization requires Hopper/Ada GPUs: on Ampere (sm 86, e.g. RTX 3090), vLLM's Marlin FP8 MoE kernel cannot tile this model's expert shape (K=352) within Ampere's 99 KB shared memory ( Invalid thread config ), and the Triton FP8 MoE backend rejects the per channel × per token scheme. INT8 W8A8 routes through vLLM's Triton Int8 MoE backend, which runs on Ampere and newer — this checkpoint serves on 2× RTX 3090. Quantization Weights: int8, symmetric, per output channel (RTN, memoryless minmax ) Activations: int8, symmetric, per token, dynamic (computed at runtime, nothing stored) Format: compressed tensors , int quantized Ignored (kept bf16): lm head (tied), embeddings, MoE routers, vision tower, self conditioning — same ignore list as the FP8 dynamic release Coverage: 11,725 quantized weight tensors (all 128 experts × 48 layers split per expert + dense MLP/attention projections), identical tensor layout to the FP8 dynamic r…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy