Gemma 4 26B A4B IT FP8 Dynamic Norouter Production ready offline FP8 checkpoint for vLLM — 47% less VRAM, 80% more concurrency vs BF16. We searched for a usable offline FP8 checkpoint of Gemma 4 26B A4B it but couldn't find one that worked cleanly with vLLM. So we vibe coded our own and are sharing it with the community. This repository hosts an offline FP8 checkpoint derived from google/gemma 4 26B A4B it for vLLM serving. No on the fly quantization needed at startup. Published by Largitdata Inc. Note: This is a derived operational checkpoint, not an official Google release. The original model's license terms, safety guidance, and documentation remain authoritative. 📖 中文說明 / Chinese Version Model Details Base model: google/gemma 4 26B A4B it Derived format: offline FP8 checkpoint for vLLM Quantization tool: llmcompressor Quantization method: FP8 DYNAMIC Calibration data: None required (dynamic quantization) Excluded weights: norm class 1D tensors — excluded to avoid expected 2D linear weight validation errors during quantization re:. router\.proj$ — MoE router weights excluded to maintain compatibility with the Gemma4 vLLM loading path Output directory name: gemma 4 26B A4B it FP…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy