Qwen3.6 27B NVFP4 NVFP4 quantized version of Qwen/Qwen3.6 27B by vrfai using llm compressor. Tested and deployed on 2× NVIDIA RTX 5090 with full tensor parallel inference via vLLM. NVFP4 Quantization Details Base model Qwen/Qwen3.6 27B Quantization NVFP4 — weights FP4, activations FP4 (dynamic local), scales FP8 Format compressed tensors (native vLLM support) Tool vllm project/llm compressor Requires NVIDIA Blackwell GPU (SM 120+), vLLM ≥ 0.19 What's Quantized / What's Not The quantization strategy carefully preserves the most sensitive components in BF16 while aggressively compressing the compute heavy stable layers: Component Precision Reason FFN / MLP — all 64 transformer layers NVFP4 High parameter density, stable under quantization Full attention projections (q/k/v/o) — 16 GQA layers NVFP4 Standard attention, tolerant to 4 bit DeltaNet / Linear attention projections — 48 layers BF16 Gated linear recurrence is sensitive to numerical errors Vision encoder — all 27 blocks + merger BF16 Vision tower preserved to maintain multimodal quality lm head BF16 Output logits preserved for generation stability The architecture of Qwen3.6 27B interleaves 3 × DeltaNet (linear attention) layer…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy