Cerebras Publishes Model Compression Policy: Unpruned Weights, Storage-Only Quantization

Cerebras has documented how it handles model compression across its public inference endpoints, and the short version is a pitch for transparency. Every model on its shared API is served in its original, unpruned form. The company does research on pruning—specifically REAP (Router-weighted Expert Activation Pruning), which permanently removes layers or experts and thereby produces a different model—but those pruned variants are published only to Hugging Face for research, never routed through the production API. Cerebras also commits to not altering a served model’s architecture without notice; any future compression would ship under separate, explicitly named endpoints.

The one compression technique Cerebras does apply is selective, weight-only quantization, and only at the storage layer. Weights are held in mixed 16/8/4-bit precision in line with common industry practice, but layers flagged as quality-sensitive are stored at full precision and dequantized on the fly, so the actual math runs in high precision. Activations, attention, and the KV cache stay full-precision and unquantized throughout.

The practical takeaway for anyone building on hosted inference: quantization changes numeric precision without touching architecture, while pruning changes the model itself—and Cerebras is drawing a hard line between the two on its public API. Note that the linked documentation covers this compression policy rather than any specific model launch or throughput benchmark; the referenced Qwen deployment and 1,500 tokens/s figure are not substantiated by the page’s content.