Ornith 1.0 35B MTPLX A multi token prediction (MTP) graft onto deepreinforce ai/Ornith 1.0 35B, packaged for MTPLX native speculative decoding on Apple Silicon. Ornith 1.0 35B (qwen35moe, 35B A3B, Qwen3.5 base) is a strong agentic coding MoE but ships without MTP heads . This build grafts the official 1 layer MoE MTP from Qwen/Qwen3.5 35B A3B (the upstream base of Ornith — dimensions match exactly) and quantizes the MTP to 4 bit , which is what makes the speedup practical (see below). Performance (M3 Max, measured) Mode tok/s Speedup MTP acceptance AR (no MTP) 76.3 1.00× — MTP depth 1 103.6 1.36× 89.6% MTP depth 2 114.9 1.50× 93.1% / 78.2% MTP depth 3 116.4 1.53× 91.5% / 80.3% / 65.6% verdict: mtp depth wins · MTPLX inspect tier: verified Why 4 bit MTP (not bf16) A bf16 graft of this same MTP layer was a net ~20× slowdown despite 92% acceptance — the MTP is a full 256 expert MoE layer, and at bf16 its draft forward costs ~67–143 ms/token (it doesn't hit the fast MoE kernel). Quantizing the MTP experts to 4 bit affine drops the draft cost to ~2.5 ms/token (~27–51×) with negligible acceptance loss, flipping the result to a real 1.53× speedup . The bottleneck was draft cost, not accep…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy