WASTE streams a 2.78T-parameter model off disk to run K3 on a 64GB laptop

WASTE is a dependency-free inference engine written in C that runs the full open-weights Kimi K3 model — 2.78 trillion parameters, 982 GiB after conversion — on a single 64GB MacBook Pro. The trick exploits how mixture-of-experts models work: only about 4% of the weights fire on any given token, so WASTE keeps the model’s shared trunk resident in RAM, streams the handful of experts each token needs directly from disk, and uses whatever memory is left as a bounded expert cache. Experts are stored in residual vector quantization at 3 bits per weight and laid out so routing to one costs exactly a single read. The result is correct but slow: roughly half a token per second, with every layer validated against a PyTorch reference to within a few parts in ten million.

The authors are candid that speed is the wrong metric here. Their claim is feasibility — they say they found no other published demonstration of a trillion-scale model streaming from disk on consumer hardware, with the best-documented 671B recipes still assuming a server stuffed with a terabyte of DDR5. Storage bandwidth, not compute, is the real constraint: each token reads about 17GB of experts, which streams fine off an internal NVMe drive at ~12.8 GB/s but crawls to thirteen seconds per token over a USB enclosure. Sixty-four gigabytes of RAM and a fast internal SSD are the practical floor; a smaller Kimi-Linear-48B variant runs from a 19GB container at a far more usable 10.7 tok/s for anyone wanting to try the engine before committing a terabyte of disk to K3.

The significance is less about K3 specifically than about what it implies. The format and engine aren’t tied to any one model, and the pitch is a frontier-scale model that answers with no network call, no per-token bill, and no data leaving the machine — the gap between “you may not send that data to an API” and “run it locally.” Notably, the project documents its own failed optimizations and records wrong numbers as wrong rather than quietly fixing them, which lends its performance claims unusual credibility.