What Cerebras does to the weights it serves

Cerebras stores model weights in a mix of 16-bit, 8-bit and 4-bit precision, but keeps the activations, attention and KV cache unquantized at full precision. Sensitive layers are held at full precision too, dequantized on the fly so the actual math runs in high precision. That is the entirety of what the company calls compression on its Shared Inference tier: selective, weight-only quantization applied at storage, and nothing else.

That sentence matters because “we host Qwen” or “we host Llama” tells you almost nothing about what a hosted inference API actually serves. Two providers can advertise the same model and hand you weights that behave differently, because somewhere between the open release and the endpoint they quantized, pruned, or otherwise altered it. Cerebras’s model overview page is an attempt to say precisely which of those it does and does not do.

Quantization and pruning are different operations. Quantization reduces the precision of the numbers that represent the weights, for example FP16 down to FP8; it shrinks memory without touching the architecture. Pruning permanently removes parts of the model, such as layers or experts, which changes the architecture and produces a genuinely different model. Cerebras does the first, at storage only, and states it does not do the second in production.

On the quantization side, the stored format is described as partial 16/8/4-bit, in line with industry norms, with sensitive layers kept at full precision and dequantized when needed. Everything outside the weights, meaning activations, attention and the KV cache, stays at full precision. The lossy step is confined to how the less-sensitive weights sit on disk; they are dequantized on the fly, so the computation that depends on them runs in high precision.

On pruning the statement is flat: none of the Shared Inference models are pruned. They are the original, unpruned community releases. Cerebras does run pruning research under the name REAP (Router-weighted Expert Activation Pruning) and publishes those pruned models on Hugging Face for research and experimentation, but they do not go through the production API.

The part worth holding Cerebras to is the promise about model IDs. It says it will serve the original weights for an existing model ID without modification and will not change a model’s architecture without notice, and that if it adopts pruning or other compression later it will ship those as separate variants with distinct model IDs. A model ID is a contract. The way you lose trust in a hosted API is by silently swapping what sits behind a stable identifier, and the benchmarks you ran last month stay valid only if the thing behind the ID has not moved.

If you depend on this, pin to explicit model IDs and treat any new variant ID as a fresh model to re-evaluate rather than a drop-in replacement. The full-precision activations and KV cache are the detail to note for long-context work, where quantized caches have a habit of quietly degrading quality; here the cache is stated to be unquantized. The pruned research models, if you want to measure the gap yourself, are in the Cerebras REAP collection on Hugging Face, not something you reach by accident through the API.

For completeness on access: the Shared Inference models run on the Free Trial and Pay as You Go tiers under rate limits and pricing, with Dedicated Inference covering reserved capacity, higher throughput, additional model families and production SLAs.