Qwen3 8B P EAGLE Drafter (Qwen3 Architecture) P EAGLE (Parallel Eagle) speculative decoding drafter for Qwen/Qwen3 8B using the Qwen3 draft architecture ( draft arch qwen3 ). The Qwen3 architecture uses additional q norm / k norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See speculators 563 for the RFC and experimental results. Benchmark Results Evaluated on RedHatAI/speculator benchmarks (9 subsets) via vLLM throughput mode. Acceptance lengths Subset k=5 k=7 : : : : HumanEval 3.51 3.73 Math Reasoning 3.73 4.04 QA 2.81 3.01 Question 2.99 3.27 RAG 3.06 2.98 Summarization 2.72 2.55 Tool Call 2.71 2.97 Translation 2.80 2.81 Writing 3.00 3.37 Average 3.04 3.19 Comparison vs Llama arch baseline Format: delta vs Llama arch P EAGLE. Subset k=5 k=7 : : : : HumanEval +0.50 +0.23 Math Reasoning +0.46 +0.22 QA +0.41 +0.19 Question +0.36 +0.24 RAG +0.57 +0.05 Summarization +0.69 +0.07 Tool Call +0.26 +0.10 Translation +0.54 0.07 Writing +0.37 +0.31 Average +0.46 (+18%) +0.15 (+5%) Qwen3 arch outperforms Llama arch on all 9 subsets at k=5 and 8/9 at k=7. Training Parameter Value Target model Qwen/Qwen3 8B Draft archi…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy