Parameter count is the wrong number to watch

PrismML’s new Ternary Bonsai 2 27B is a 27-billion-parameter multimodal model that ships as a 5.9GB file. Its full-precision counterpart, Qwen3.8 27B, needs more than nine times that. The parameter count is identical; the memory you have to allocate to run it is not.

The mechanism is ternary quantization. Instead of storing each weight as a 16-bit float, Bonsai 2 represents every weight as one of three values, −1, 0, or +1, with FP16 scaling factors shared across groups of weights. PrismML puts the effective rate at 1.76 bits per weight, which includes the overhead of those group scaling factors. The low-bit representation runs end to end across the language model rather than on a handful of layers. The released model supports a 262K-token context window, takes text and image input, and carries an Apache 2.0 license.

Compression this aggressive normally costs capability, and the useful question is where. PrismML reports an aggregate benchmark score of 83.9 in thinking mode, which it puts at 98.2% of the full-precision model’s aggregate performance, up from 95% in the first Bonsai 27B release two months ago. Those are the vendor’s own numbers, measured across a suite covering reasoning, math, coding, instruction following, vision, and agentic tool use. The company argues retention holds specifically in the areas where quantization error tends to compound, such as coding-agent loops, long-horizon tool use, and multimodal workflows, where a small per-step error can snowball across many steps.

The energy numbers make the point most directly. On an RTX 4090, PrismML measures 0.714 mWh per token and says that makes the 27B ternary model 40% more energy-efficient per token than an 8B model running in full precision. A model with more than three times the parameters drawing less power per token is the whole argument for why parameter count tells you less than the footprint does. Throughput lands at up to 143 tokens/second on an RTX 5090 and 46.8 tokens/second on an M5 Max.

For anyone deciding what to run locally, this shifts the comparison. A 5.9GB model fits in the VRAM of a mid-range consumer GPU or in the unified memory of a laptop, so workloads you would otherwise send to a cloud API can stay on the machine. Those are the cases where a capable local model changes the data-flow picture and not only the bill: private document analysis, code that should not leave the building, high-frequency agent loops where round-trip latency and per-call cost accumulate. PrismML frames the intended pattern as hybrid orchestration, with local models handling the sensitive or high-frequency work and escalating to the cloud selectively.

Two cautions before you treat 98.2% as settled. The retention figure is aggregate and self-reported, so it averages over benchmarks that may look nothing like your workload. And an aggregate number hides its distribution: a model can hold its average while losing ground on the one capability you depend on. Run your own evals on the task you actually care about.

Bonsai 2 27B runs on NVIDIA GPUs through CUDA and on Mac, iPhone, and iPad through MLX, using custom low-bit kernels on both. The weights are available now under Apache 2.0, which means you can test the retention claim against your own tasks instead of taking it.