Qwen3.5 9B AWQ Model Highlights This model Qwen3.5 9B AWQ was converted to AWQ format, from Qwen/Qwen3.5 9B using llm compressor version 0.10.0.1 (https://github.com/vllm project/llm compressor.git). With using dataset wikitext from Salesforce/wikitext & COCO Caption2017 from lmms lab/COCO Caption2017 Datasets: Salesforce/wikitext lmms lab/COCO Caption2017 Base Model: Qwen/Qwen3.5 9B Use with VLLM 1. Download models at first by using hf 2. Copy hash for snapshots directory, then use it for chat templates and tool call parser ex: /home/xxx/.cache/huggingface/hub/models nicklas373 Qwen3.5 9B AWQ/snapshots/HASH CODE/xxx 3. Run models with this command Notes: Already fixed for vision models :D ~~For now only text model that capable to use, while vision model aren't work properly (Gilberish output while run with VLLM)~~ ~~Current VLLM serve script already adjusted to disable vision model, so it can use normally with text only :D~~ Qwen3.5 9B [!Note] This repository contains model weights and configuration files for the post trained model in the Hugging Face Transformers format. These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc. Over recent m…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy