Qwen3.6 35B A3B Q4 K M GGUF Quantization This repository contains the Q4 K M GGUF format of the Qwen3.6 35B A3B model. These files were quantized by Abiray using llama.cpp to make the model accessible for consumer hardware and CPU heavy environments. 📦 Other Available Formats I have processed this model into several different quantization formats. You can find them in my other repositories: Abiray/Qwen3.6 35B A3B Q8 0 GGUF (High quality, largest file) Abiray/Qwen3.6 35B A3B Q6 K GGUF (Very high quality) Abiray/Qwen3.6 35B A3B Q5 K M GGUF (Great balance of quality and size) Abiray/Qwen3.6 35B A3B Q4 K M GGUF (Good quality, recommended for most users) Abiray/Qwen3.6 35B A3B Q3 K M GGUF (Low quality, smallest file) 💻 How to run with llama.cpp You can run this model locally using llama cli from the llama.cpp project. bash Example command (adjust threads and context size to your machine) ./llama cli m Qwen3.6 35B A3B Q4 K M.gguf p "Your prompt here" n 512 t 8 c 4096
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy