MTP BF16 graft for vLLM speculative decoding This checkpoint is based on cyankiwi/Qwen3.6 27B AWQ BF16 INT4 version 26.05.01 at commit 232ed78ba2a5eba6c39cd36d1e11ff7347e0883e . The cyankiwi checkpoint is an AWQ / compressed tensors INT4 quantization of Qwen/Qwen3.6 27B . The upstream 26.05.01 checkpoint still stores the MTP projection tensors as compressed tensors INT4 entries. vLLM MTP speculative decoding expects the MTP modules to load as normal raw tensors, so this variant keeps the main model quantized but replaces only the mtp. tensors with raw BF16 tensors from the original Qwen/Qwen3.6 27B checkpoint. What changed in this repository: Started from cyankiwi version 26.05.01 / commit 232ed78ba2a5eba6c39cd36d1e11ff7347e0883e . Removed the quantized MTP entries: mtp. .weight packed , mtp. .weight scale , mtp. .weight zero point , and mtp. .weight shape . Added the 15 raw BF16 mtp. tensors from Qwen/Qwen3.6 27B . Rewrote model 00006 of 00006.safetensors to contain the BF16 MTP tensors. Added the MTP linear modules to quantization config.ignore so they are treated as unquantized BF16 modules. Recomputed model.safetensors.index.json metadata.total size from the actual output tenso…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy