Introduce We adapted the official speculative sampling training method, Eagle3, for training on Qwen3 30B A3B After implementing Eagle3, the inference performance of Qwen3 30B Moe using the SGLang framework on 8 H200 GPU improved from 183 tokens/s to 325 tokens/s. The TPS (tokens per second) improvement reached nearly 70%. On a single RTX 5090, the TPS (transactions per second) of Qwen3 8B Eagle3 increased from 164 to 268. model gpu tps qwen3 30b moe h200 147 qwen3 30b moe eagle3 h200 231 qwen3 30b moe 8 h200 183 qwen3 30b moe eagle3 8 h200 325 qwen3 30b moe 8 5090 164 qwen3 30b moe eagle3 8 5090 268 Join our AI computing power cloud platform now and enjoy the best AI cloud service experience. The link is as follows: https://tenyunn.com/ How to use To use Eagle3 with SGLang, first replace the qwen3 moe.py file in SGLang’s directory (sglang/python/sglang/srt/models/) with the qwen3 moe.py file from this project. The launch command for using Eagle3 with SGLang is: How to train Training Dataset: ultrachat 200k. Only the prompts from these datasets were utilized for data synthesis. This synthesized data is used to train the Eagle modules. dataset nums: 600K samples,1B tokens Evaluation…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy