GitHub Repo Technical Report 👋 Join us on Discord and WeChat Overview BitCPM CANN 0.5B unquantized is the unquantized QAT (Quantization Aware Training) checkpoint of BitCPM CANN 0.5B, designed for continued pre training and fine tuning . It preserves full precision latent weights with ternary fake quantizers (weights → { 1, 0, 1} with group wise scaling, trained via STE) defined in modeling.py , enabling the model to keep learning under quantization constraints. For technical details, see our Technical Report. ⚠️ This model is NOT for direct inference. For inference, use the pseudo quantized version: openbmb/BitCPM CANN 0.5B. Continued Pre training & Fine tuning The only requirement is that the forward pass must go through the bundled modeling.py (which contains the ternary fake quantizer). Load with trust remote code=True and do NOT replace or bypass the model's forward logic. Option 1: DeepSpeed (Recommended) We provide ready to use training scripts in the example directory (using the 1B model as an example): Continued pre training : example/run.sh + example/train.py SFT (Supervised Fine tuning) : example/run sft.sh + example/train sft.py Quick start: Option 2: HuggingFace compa…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy