Qwen3.6 27B AWQ 4bit This repository contains an optimized, 4 bit quantized version of the original Qwen3.6 27B model by Alibaba Cloud. The quantization was performed using the AWQ algorithm via the llmcompressor library, drastically reducing the VRAM required for local inference without a perceptible degradation in the original model's accuracy. Quantization Details Base Model: Qwen/Qwen3.6 27B Algorithm: AWQ Scheme: W4A16 ASYM (4 bit asymmetric weights, 16 bit activations) Ignored Modules: lm head , visual , vision (Vision encoder layers are kept in their original precision to maintain multimodal capabilities). Weights Format: Safetensors Calibration Dataset: HuggingFaceH4/ultrachat 200k (512 random samples from the train sft split, formatted with the model's chat template. Max sequence length: 1024). Hardware Requirements Estimated VRAM: ~18 GB (Compared to the ~54 GB required by the FP16 model). Compatible with high performance inference engines such as vLLM . Below is the original model card provided by the Qwen developers. Qwen3.6 27B [!Note] This repository contains model weights and configuration files for the post trained model in the Hugging Face Transformers format. Thes…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy