Qwen3.6 27B FP8 FP8 (W8A8) quantized version of Qwen/Qwen3.6 27B by vrfai using llm compressor. Also available: vrfai/Qwen3.6 27B NVFP4 — more aggressive quantization for Blackwell GPUs only. FP8 Quantization Details Base model Qwen/Qwen3.6 27B Quantization W8A8 FP8 — weights FP8 static, activations FP8 static Strategy tensor (per tensor symmetric, memoryless minmax) Format compressed tensors (native vLLM support) Tool vllm project/llm compressor Requires NVIDIA Ampere / Hopper / Blackwell (SM 89+) What's Quantized / What's Not Same selective strategy as the NVFP4 variant — sensitive components are preserved in BF16: Component Precision Reason FFN / MLP — all 64 transformer layers FP8 High parameter density, stable under quantization Full attention projections (q/k/v/o) — 16 GQA layers FP8 Standard attention, tolerant to 8 bit DeltaNet / Linear attention projections — 48 layers BF16 Gated linear recurrence sensitive to numerical errors Vision encoder — all 27 blocks + merger BF16 Vision tower preserved for multimodal quality lm head BF16 Output logits preserved for generation stability Quantization Config (llm compressor) Quick Start (vLLM) Single GPU (≥ 24 GB VRAM, SM 89+): Quanti…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy