Qwen 3.6 35B A3B NVFP4 Fast GGUF About This model is an unmodified GGUF quantization of unsloth/Qwen3.6 35B A3B NVFP4 Fast, made with the llama.cpp conversion tool. Please refer to unsloth repository for quality metrics. Your feedback, contributions, optimization proposals and questions are highly appreciated. This GGUF is primarily intended to be run fully on the GPU without offloading to the CPU. However, you can absolutely run it with offloading. The relevant guides are provided in this repo. Speed metrics I've tested this model on my setup (5070 Ti 16GB, 5060 Ti 16GB, 32GB DDR5). Token generation and prefill speeds will vary depending on your hardware. Please use the metrics below as a rough estimate of your potential performance. Setup Token generation, t/s Prompt prefill, t/s : : : 5070ti + 5060ti ~134.4 ~3462 5070ti + CPU ~68.7 ~324 5060ti + CPU ~51.0 ~161 Quick start This GGUF requires a recent build of llama.cpp. If the commands below raise a compatibility error, please update your llama.cpp version. All commands have been tested on llama.cpp version b9957 ensure your version is =b9957 . Agentic tasks Speed efficient The token generation speeds listed below were achieved u…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy