BitNet b1.58 2B4T Scaling Native 1 bit LLM This repository contains the weights for BitNet b1.58 2B4T , the first open source, native 1 bit Large Language Model (LLM) at the 2 billion parameter scale, developed by Microsoft Research. Trained on a corpus of 4 trillion tokens, this model demonstrates that native 1 bit LLMs can achieve performance comparable to leading open weight, full precision models of similar size, while offering substantial advantages in computational efficiency (memory, energy, latency). ➡️ Technical Report: BitNet b1.58 2B4T Technical Report ➡️ Official Inference Code: microsoft/BitNet (bitnet.cpp) Model Variants Several versions of the model weights are available on Hugging Face: microsoft/bitnet b1.58 2B 4T : Contains the packed 1.58 bit weights optimized for efficient inference. Use this for deployment. microsoft/bitnet b1.58 2B 4T bf16 : Contains the master weights in BF16 format. Use this only for training or fine tuning purposes. microsoft/bitnet b1.58 2B 4T gguf (This repository): Contains the model weights in GGUF format, compatible with the bitnet.cpp library for CPU inference. Model Details Architecture: Transformer based, modified with BitLinear lay…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy