Paper Link 👁️ 🏠 Home Page       🤗 Model       📊 Dataset       📄 Paper       🚀 Demo    1. Introduction OpenCoder is an open and reproducible code LLM family which includes 1.5B and 8B base and chat models, supporting both English and Chinese languages. Starting from scratch, OpenCoder is pretrained on 2.5 trillion tokens composed of 90% raw code and 10% code related web data, and supervised finetuned on over 4.5M high quality SFT examples, finally reaching the performance of top tier code LLMs. We provide not only model weights and inference code, but also the reproducible training data, the complete data processing pipeline, rigorous experimental ablation results, and detailed training protocols. Empowering researchers to build and innovate, OpenCoder is your open foundation for advancing code AI. Complete Open Source : OpenCoder ensures full transparency by releasing not only the model weights and forthcoming inference code but also the complete data cleaning code for training. This release includes high quality synthetic data, an extensive set of checkpoints, and a dataset of over 4.5 million supervised…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy