Cloudflare prototype compresses cache with Zstandard, reclaiming petabytes

Facing steep RAM and disk price hikes, Cloudflare built a prototype called Cache Transcoding that compresses eligible assets with Zstandard (zstd) as they enter the cache. The encoding happens once inside its Pingora proxy on a cache miss; the asset stays compressed on disk and while moving between data centers via Tiered Cache, and is decoded back to its original form only on the final hop to the client. In early testing, eligible assets shrank to roughly a third of their on-disk size, buying petabytes of effective capacity and cutting cross-datacenter backbone traffic in exchange for a few percent of extra CPU.

The system is deliberately selective. Already-compressed media like images, video, and fonts (21% of requests but 63% of bytes in Cloudflare’s sample) is skipped, since recompressing it wastes CPU for no gain. Instead, transcoding targets compressible text — HTML, JSON, CSS, and JavaScript — that arrives uncompressed, which made up most requests but a smaller share of bytes and compressed about 2.8x. The prototype uses zstd level 3 and only touches 200 OK responses with no existing Content-Encoding, a compressible content type, and a known length of at least 4 KiB, a threshold that discards many tiny objects while excluding only about 1% of otherwise-eligible bytes.

A notable finding: limiting the feature to popular “hot” content didn’t pay off, because decoding cost is incurred on every hit while storage savings scale with total objects stored — so compressing all eligible text was both simpler and more effective. A metadata marker records the compressed representation and prevents double-encoding as objects move between cache tiers. The compression level and size threshold are tunable parameters, leaving room to push for higher ratios once the CPU budget is better understood. The work, built during a Cloudflare internship, was validated across more than a million requests spanning cache misses, hits, and single- and multi-tier fills.