Qwen3.6-27B-PRISM-PRO — DQ GGUF

llama.cpp-native GGUF quantization of Qwen3.6-27B-PRISM-PRO using the PRISM project's dynamic-quant (DQ) recipe. ~13.7 GB (vs 55 GB BF16).

PRISM-PRO of Qwen/Qwen3.6-27B (bias/propoganda removal) This GGUF preserves the model's native MTP draft head + full vision tower, and pairs with the separately-published EAGLE-3 drafter for lossless faster decode.

Performance

llama.cpp on a single NVIDIA Blackwell GPU, single-stream greedy decode:

Speculative decoding is lossless (output token-identical to non-spec greedy, modulo batched-verify floating-point non-associativity intrinsic to all spec decoding). For a faster SGLang deployment (~183 tok/s, ~1.97× over no-spec) using the BF16 target + EAGLE-3, see the drafter repo.

Quick start (llama.cpp)

# 1. no-spec baseline
./llama-server --model Qwen3.6-27B-PRISM-PRO-DQ.gguf

# 2. native MTP speculative decoding (the model's own draft head -- fastest in llama.cpp)
./llama-server --model Qwen3.6-27B-PRISM-PRO-DQ.gguf \
    --spec-type draft-mtp --spec-draft-n-max 1 --spec-draft-n-min 1

# 3. EAGLE-3 chain (needs the WIP PR #18039 patches + the RS-rollback fix --
#    a one-shot llama.cpp patch script is documented alongside the drafter:
#    https://huggingface.co/Ex0bit/Qwen3.6-27B-PRISM-EAGLE3)
./llama-server --model Qwen3.6-27B-PRISM-PRO-DQ.gguf \
    --spec-type draft-eagle3 --model-draft <eagle3-drafter.gguf> \
    --spec-draft-n-max 2

Provenance

• Base: Qwen/Qwen3.6-27B (hybrid: 48 GatedDeltaNet linear-attention layers
  • 16 full-attention layers; hidden 5120; vocab 248 320; native MTP head).
• PRISM Dynamic Quantization: PRISM DQ recipe (llama.cpp GGUF dynamic quant) — preserves the MTP draft head (15 tensors) and the full vision tower (333 tensors).

License

Apache-2.0. Derived from Qwen/Qwen3.6-27B (Apache-2.0).