Qwen3.6 27B GPTQ 8 bit GPTQ 8 bit quantization of Qwen/Qwen3.6 27B, a 27B parameter dense multimodal model. Includes the full vision encoder and MTP (Multi Token Prediction) module for image understanding and speculative decoding support. Model Overview Architecture : Qwen3 5ForConditionalGeneration (multimodal: text + vision; dense sibling of qwen3 5 moe ) Total parameters : ~27B Layers : 64 (48 linear attention + 16 full attention, repeating 3:1 pattern) Hidden size : 5120, intermediate size: 17408 (dense MLP — no MoE) Context length : 262,144 tokens Vision encoder : 27 block ViT, BF16 (333 tensors) MTP module : 1 layer speculative decoding head, BF16 (15 tensors) Quantization Details All quantizable Linear modules in the text decoder are quantized to INT8 using GPTQ. The vision encoder, MTP module, norms, embeddings, and LM head remain at BF16/FP16 for quality preservation. Component Precision Notes mlp.{gate proj, up proj, down proj} INT8 (GPTQ) All 64 layers self attn.{q,k,v,o} proj INT8 (GPTQ) 16 full attention layers linear attn.{in proj qkv, in proj z, out proj} INT8 (GPTQ) 48 linear attention layers (GatedDeltaNet) linear attn.{in proj a, in proj b} FP16 Tiny projections,…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy