Qwen3.6 27B GPTQ 4 bit GPTQ 4 bit quantization of Qwen/Qwen3.6 27B, a 27B parameter dense multimodal model. Uses FOEM (First Order Error Matters, AAAI 2026) for outlier aware error compensation, enabling pure 4 bit quantization without per module FP16 exclusions. Includes the full vision encoder and MTP (Multi Token Prediction) module for image understanding and speculative decoding support. Model Overview Architecture : Qwen3 5ForConditionalGeneration (multimodal: text + vision; dense sibling of qwen3 5 moe ) Total parameters : ~27B Layers : 64 (48 linear attention + 16 full attention, repeating 3:1 pattern) Hidden size : 5120, intermediate size: 17408 (dense MLP — no MoE) Context length : 262,144 tokens Vision encoder : 27 block ViT, BF16 (333 tensors) MTP module : 1 layer speculative decoding head, BF16 (15 tensors) Quantization Details All quantizable Linear modules in the text decoder are quantized to INT4 using GPTQ + FOEM. The vision encoder, MTP module, norms, embeddings, and LM head remain at BF16/FP16 for quality preservation. Component Precision Notes mlp.{gate proj, up proj, down proj} INT4 (GPTQ + FOEM) All 64 layers self attn.{q,k,v,o} proj INT4 (GPTQ + FOEM) 16 full…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy