Bonsai 2 27B: a 27B model in 5.9GB that keeps 98% of its benchmarks

PrismML has released Ternary Bonsai 2 27B, a heavily quantized version of Qwen3.8 27B that stores each weight as a ternary value (−1, 0, +1) with FP16 group-wise scaling. That works out to roughly 1.76 bits per weight and a 5.9GB total footprint — about 9x smaller than the full-precision original — while retaining 98.2% of aggregate benchmark performance (a score of 83.9). The model keeps a 262K-token context window and text-plus-image input, and ships under Apache 2.0 with weights available now.

The more interesting claim is where the accuracy survives. Low-bit quantization usually shows its damage first in coding agents, tool use, and long-horizon workflows, where small per-step errors compound. PrismML says Bonsai 2 holds up in exactly those areas, which is what makes a local-only model plausible for real work: coding-agent loops, computer-use tasks, and private document or multimodal analysis that never leaves the device. Reported throughput is up to 143 tokens/sec on an RTX 5090 and 46.8 on an M5 Max, with energy use of 0.714 mWh/token on a 4090 — pitched as 40% more efficient than an 8B model at full precision. It runs on NVIDIA via CUDA and Apple hardware via MLX through custom low-bit kernels.

Compared with the first Bonsai 27B, the retention gap versus full precision has narrowed from about 95% to over 98%, close enough that PrismML calls it practically lossless. The broader argument is that low-bit representation isn’t just a trick for squeezing models onto laptops — it changes the economics everywhere, letting datacenters fit bigger models in the same memory, serve more users per GPU, and split workloads dynamically between local and cloud. The framing to watch is intelligence delivered per unit of memory, compute, and power, rather than raw capability alone. PrismML, a Caltech-spun startup backed by Khosla Ventures, Cerberus, Google, and Samsung, positions compression as a core deployment strategy rather than an afterthought.