Qwen3.5 9bxOpus 4.7 sonic GGUF : GGUF A distilled 9B code/reasoning model in GGUF format, optimized for local inference via llama.cpp. Quick Start Server Launch Recomment configure Using mmproj is more like to Multi Token Prediction( MTP ) reason for support text, image, video, audio and pdf, is an advanced training and inference technique for memory and cache. I recomment to use ctk q8 0 ctv q8 0 for better cache and load. Preset Configs Mode Flags Coding temp 0.4 top p 0.9 top k 50 repeat penalty 1.1 Reasoning temp 0.6 top p 0.95 top k 40 Low VRAM c 32768 n 4096 flash attn off ngl 20 Key Specs Base: Qwen3.5 9B · Format: Q4\ K\ M GGUF Context: 32K–64K practical · Output: up to 8K tokens KV cache: 8 bit (q8\ 0) · GPU: full offload ( ngl 1 ) Thinking mode: optional — improves reasoning, adds latency Caveats Quality drops beyond 64K context 9B class — sensitive to sampling parameters Thinking mode can produce unstable output if misaligned with training
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy