OPDLM 0.6B OPDLM 0.6B is a block diffusion language model (DLM) obtained by post training an autoregressive language model (ARLM) into a diffusion language model via on policy distillation . arXiv report: arxiv.org/abs/2606.06712 Highlights Converted, not pretrained from scratch: built from a strong ARLM, reusing its prior. Training efficient: orders of magnitude fewer tokens than from scratch DLM training (same base ARLM). Inference efficient: parallel token decoding via block diffusion. Model Details Developed by: DIVE Lab, Texas A&M University Base model: Qwen3 0.6B Model type: Block diffusion language model (decoder based) Block size: 4 Parameters: ~0.6B Language: English License: MIT Training Method: On policy distillation from a frozen ARLM teacher into a block DLM student. Conversion budget: ~ B tokens Data: opdlm train data Results For detailed results and benchmarks, please refer to our paper: arxiv.org/abs/2606.06712 Citation
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy