Qwen3.6 27B A3B Coder A code specialist expert prune of Qwen3.6 35B A3B: the MoE is reduced from 256 experts to 184 (72 dropped per layer, ~35B→27B, still A3B active) using a code targeted competence map (LiveCodeBench + MultiPL E competence classes). Same router, attention, norms, MTP head and vision tower as the base — only the expert keep set changes. Served at top 10 ( num experts per tok = 10 , baked as the default). This is a routing recovery lever: after pruning to 184 experts, activating the top 10 (vs the base top 8) recovers instruction following at no cost to code (see below). No fine tuning, no distillation — pure expert selection + a routing width dial. Recipe 1. Competence map : the 256e teacher is profiled per expert on a balanced corpus + targeted LiveCodeBench and MultiPL E (Rust/Java/JS) PASS response classes. 2. Drop map : wmax aggregation with the LCB + MPE classes up weighted (1.5) → 72/256 experts dropped per layer, protecting the code competent experts. 3. Top 10 routing ( num experts per tok = 10 ) baked into the config → the shipped default. Pass override kv qwen35moe.expert used count=int:8 to any llama.cpp tool to A/B back to native top 8. Evaluation (Q6…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy