gpt oss 20b DFlash Paper GitHub Blog DFlash is a novel speculative decoding method that utilizes a lightweight block diffusion model for drafting. It enables efficient, high quality parallel drafting that pushes the limits of inference speed. This model serves as the drafter component and contains 0.8B parameters . It must be used in conjunction with the target model openai/gpt oss 20b . 📊 Training Data gpt oss 20b DFlash is trained on 800K samples , drawn from: nvidia/Nemotron Post Training Dataset v2 theblackcat102/evol codealpaca v1 For all samples, the response portion was regenerated using the target model openai/gpt oss 20b . 🚀 Quick Start SGLang Installation Launch Server Usage vLLM Installation Launch Server Usage Evaluation We use a block size of 8 (7 draft tokens) during speculation. DFlash consistently achieves high acceptance lengths and speedups across different concurrency levels. All experiments are conducted using SGLang on a single H200 GPU . The numbers reported are end to end speedup (including prefill time). You can specify different block size during inference by passing speculative num draft tokens arguments when launch the server. The reasoning effort is se…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy