Prism ML Website Whitepaper Demo & Examples Colab Notebook Discord Bonsai 4B GGUF 1bit End to end 1 bit language model for llama.cpp (CUDA, Metal, CPU) 14.1x smaller than FP16 4.2x faster on RTX 4090 runs on any device Highlights Deployed footprint fits on virtually any device with a GPU End to end 1 bit weights across embeddings, attention projections, MLP projections, and LM head GGUF Q1 0 (g128) format for 1 bit packing of weights with shared scales for each group (group size 128). Cross platform : CUDA (RTX/datacenter), Metal (Mac), Swift (iPhone/iPad), Android MLX companion : also available as MLX 1 bit g128 for native Apple Silicon inference Resources Google Colab — try Bonsai in your browser, no setup required Whitepaper — for more details on Bonsai, check out our whitepaper Demo repo — comprehensive examples for serving, benchmarking, and integrating Bonsai Discord — join the community for support, discussion, and updates 1 bit kernels : llama.cpp fork (CUDA + Metal) · MLX fork (Apple Silicon) · mlx swift fork (iOS/macOS) Locally AI — we have partnered with Locally AI for iPhone support Model Overview Item Specific…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy