Qwen3.6 27B Text NVFP4 MTP NVFP4 quantized text only sibling of Qwen/Qwen3.6 27B , with the MTP (Multi Token Prediction) head restored in bf16 so speculative decoding actually works. What's different from sakamakismile/Qwen3.6 27B NVFP4 This repo ( Text NVFP4 MTP ) Qwen3.6 27B NVFP4 Quantization format modelopt (vLLM SM120 native path) compressed tensors MTP head Restored in bf16, working Dropped during export → 0% draft acceptance Vision tower Stripped (text only) Present (kept for VLM use) Suggested launch with speculative config without speculation The original Qwen3.6 27B NVFP4 is left untouched so existing users (~15K downloads) are not disrupted. This is a focused text only sibling for users who want maximum speed and don't need vision input. Why this exists Two HF Discussion threads on the original repo prompted this: 5 — slower than official FP8 on Blackwell — root cause is the compressed tensors NVFP4 path being slower than modelopt on Blackwell SM120; this repo uses modelopt natively. 7 — MTP not responding — AutoModelForCausalLM.from pretrained does not load the MTP head, so it gets dropped during quantization, leading to 0% draft acceptance. This repo grafts the 15 mtp.…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy