kimi k2.6 eagle3 mla Model Overview kimi k2.6 eagle3 mla is an Eagle3 MTP draft model with MLA (Multi Latent Attention) for accelerating inference of Kimi K2.6, trained with TorchSpec — an online speculative decoding training framework that runs FSDP training and inference concurrently. If you find this draft model useful, please give our project TorchSpec a star on GitHub. Why an MLA (Multi Latent Attention) Draft Model Compared with an MHA draft model, the MLA variant is a better fit for Kimi K2.6 deployment: Uses less KV cache, which reduces serving memory pressure. Matches Kimi K2.6's MLA architecture, so it fits more naturally into the inference engine's KV cache handling under different serving scenarios such as PD Disaggregation. Training Setup Cluster : 3 nodes × 8× B200 (24 GPUs total) Training : 1 node (8 GPUs), FSDP Inference : 2 nodes (16 GPUs), vLLM (TP=8 per node) Continual training : Initialized from kimi k2.5 eagle3 mla checkpoint Iterations : 9,279 steps Learning rate : 2e 5, cosine schedule Performance The primary metric is accept length — the average number of tokens accepted per speculation step with num speculative tokens=3 . Higher is better. Benchmarks were r…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy