Prism ML Website White Paper Demo & Examples Discord Ternary Bonsai 8B gguf Ternary (1.58 bit) language model in GGUF Q2 0 format for llama.cpp Resources White Paper Demo repo — examples for serving, benchmarking, and integrating Bonsai Discord — community support and updates Kernels : Q2 0 is not yet in mainline llama.cpp . Use our fork at PrismML Eng/llama.cpp ( prism branch, default) which adds Q2 0 support for CPU (NEON/generic) and Metal. Upstream PR coming soon. Model Overview Item Specification : : Base model Qwen3 8B Parameters 8.19B (~6.95B non embedding) Architecture GQA (32 query / 8 KV heads), SwiGLU MLP, RoPE, RMSNorm Layers 36 Transformer decoder blocks Context length 65,536 tokens Vocab size 151,936 Weight format GGUF Q2 0 g128: { 1, 0, +1} with FP16 group wise scaling Packed Q2 0 size 2.03 GiB (2.18 GB) Ternary coverage Embeddings, attention projections, MLP projections, LM head License Apache 2.0 Quantization Format: GGUF Q2 0 (g128) Each weight takes a value from { 1, 0, +1}, with one shared FP16 scale per group of 128 weights: Q2 0 encodes each weight as a 2 bit code q in {0, 1, 2, 3} , dequantized via w = (q 1) scale…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy