z lab/Qwen3.6 27B PARO Pairwise Rotation Quantization for Efficient Reasoning LLM Inference ParoQuant is the state of the art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z lab/paroquant. z lab/Qwen3.6 27B PARO is a 4 bit Qwen/Qwen3.6 27B quantized with ParoQuant. Check out other ParoQuant models from the Hugging Face collection. Quick Start Installation Interactive Chat OpenAI Compatible API Server For vLLM, you can directly use vllm serve to serve ParoQuant models: For other frameworks: For MLX, add vlm if you wish to load the VLM components and use the model's multimodal features. For vLLM, VLM components are loaded by default and can be skipped with the server argument language model only . [!NOTE] The visual components in this checkpoint is stored in original precision, and only the language components are quantized to 4 bits; as a result, the model size is larger than a fully quantized model. Avoid loading the VLM components if you are not using the multimodal features for the best efficiency. Docker (NVIDIA GPU)…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy