As the global technology industry grapples with dramatic increases in memory and hardware expenses, infrastructure providers are forced to find innovative ways to maximize existing resources. At Cloudflare, which operates a massive, distributed storage network and content delivery network (CDN), making efficient use of deployed memory is essential for maintaining high-performance service for millions of customers. To tackle rising component costs, the company has developed and prototyped a new architectural approach designed to significantly expand effective cache capacity without requiring an immediate, massive hardware expansion.
The experimental system, dubbed Cache Transcoding, was engineered as an internship project within Cloudflare’s 1.1.1.1 Intern Program. By integrating the Zstandard compression algorithm directly into Pingora—Cloudflare’s custom-built proxy that connects its network to the wider internet—the architecture deliberately trades a minor, calculated increase in CPU usage for substantial storage and cross-data center bandwidth savings. Initial testing of the prototype demonstrates that the system can shrink eligible web assets to roughly a third of their original on-disk size on average.
The Operational Mechanics of Cache Transcoding
Traditionally, Cloudflare stores web assets using the exact content encoding supplied by their respective origin servers. If an origin server transmits an uncompressed response, the network stores those uncompressed bytes directly on disk and transfers them between data centers in that exact form. Cache Transcoding fundamentally alters this behavior by introducing compression at the cache layer itself.
When an eligible response enters the cache for the first time, the Pingora-based proxy encodes the response body using Zstandard, commonly referred to as zstd, before writing the data to disk. The cache metadata is carefully updated to record that the stored representation is compressed while preserving the original content length. This compressed format is maintained for the entire lifecycle of the asset while it resides in the cache, and it remains compressed as the asset moves across data centers via Cloudflare’s Tiered Cache architecture. Just before the response is ultimately served back to the client, the proxy decodes it back to its original identity representation.
This design ensures that the CPU cost of encoding is paid only once when an asset first enters the cache, while the corresponding storage and bandwidth savings are realized repeatedly every single time that asset is reused. Performance metrics gathered during testing indicate that encoding costs approximately 4.31 nanoseconds per byte—translating to roughly 232 megabytes per second—while decoding is notably faster at 1.56 nanoseconds per byte, or approximately 641 megabytes per second. Because assets are served to users far more frequently than they are initially fetched and filled into the cache, the asymmetry heavily favors overall system efficiency.
Understanding the Role of Zstandard
At the heart of this new architecture is Zstandard, a lossless compression algorithm originally developed by Yann Collet at Facebook and subsequently open-sourced in 2016. Because the algorithm is lossless, decoding compressed data restores every single byte so that it is entirely identical to the original file. This allows Cloudflare to alter how assets are represented on disk and transmitted across its backbone network without modifying the underlying content delivered to end users.
Zstandard was selected because of its unique ability to carefully balance a high compression ratio with rapid execution speeds. In earlier browser compression evaluations conducted by Cloudflare, zstd demonstrated the ability to compress data 42 percent faster than Brotli while achieving nearly identical file sizes. Furthermore, it produced files roughly 11.3 percent smaller than traditional gzip while maintaining a comparable speed. This operational balance is crucial for a large-scale proxy infrastructure, where transcoding touches an immense volume of web traffic and both encoding and decoding operations must execute with minimal latency.
For the initial prototype, engineers utilized zstd level 3. This specific configuration provides the vast majority of the algorithm’s potential compression benefits without turning cache fill operations into a severe CPU bottleneck.
Strategic Targeting: Why Not Everything is Compressed
A foundational principle of the Cache Transcoding project is that not all web traffic benefits equally from compression. Attempting to transcode assets that are already heavily compressed would merely burn valuable CPU cycles without yielding any meaningful storage or bandwidth returns.
An analysis of typical traffic samples reveals that media assets—such as images, videos, and fonts—account for approximately 21.4 percent of total requests but represent an overwhelming 63.3 percent of total bytes transferred. Because these binary formats are already compressed at their source, applying zstd transcoding to them would be redundant and inefficient.
Compressible text assets, however, present a starkly different profile. Formats like HTML, JSON, CSS, and JavaScript accounted for 67.3 percent of requests in the sample data while making up 22.3 percent of total bytes. Within this specific text category, approximately 71 percent of responses arrived at the network uncompressed, with the Content-Encoding header left unset, making them prime candidates for effective compression. In controlled testing environments, these eligible text assets compressed by an average factor of roughly 2.8 times.
To avoid wasting computational resources on marginal gains, the prototype enforces strict eligibility criteria. The system exclusively targets 200 OK responses where the Content-Encoding header is unset, the Content-Type indicates compressible text, and the response features a known Content-Length of at least 4 kibibytes (KiB). Slice subrequests, responses utilizing active upstream compression, range requests, precompressed data, unknown length bodies, and binary content are deliberately left untouched by the transcoding process.
The 4 KiB size threshold was established to filter out a vast number of tiny requests that would introduce unnecessary per-object metadata overhead while discarding only about one percent of otherwise eligible bytes.
Navigating Tiered Cache and Global Data Movement
Beyond local disk storage optimization, Cache Transcoding provides distinct advantages for data movement across Cloudflare’s global infrastructure via Tiered Cache. When an asset is requested across different data centers, transferring the compressed zstd representation rather than uncompressed bytes significantly reduces the volume of data moving across the company’s backbone network.
In scenarios where a lower-tier data center experiences a cache miss but an upper tier already possesses the object, the origin server remains entirely uninvolved in the transaction. The compressed object moves directly between the cache tiers, remaining compressed both on the wire and on disk, and is decoded only once at the lower tier before reaching the client. If the lower tier already holds the object locally, no network transfer or encoding computation is required at all; the system simply reads the zstd bytes from disk, decodes them on the fly, and passes the original asset onward.
To prevent redundant processing, a dedicated storage encoding marker is embedded within the cache metadata. This ensures that an object is never encoded more than once, allowing any cache layer receiving an item from another tier to recognize that it is already stored using zstd and preserve it in that optimized form.
Validating Performance Across the Network
To rigorously test the prototype before wider deployment, Cloudflare engineers subjected the architecture to a controlled test zone, correlating every action across request logs, Prometheus metrics, and Jaeger traces. The validation campaign meticulously covered cache misses, cache hits, single-hop fills, and complex Tiered Cache scenarios by varying cache keys to force requests along specific pathways.
One major performance evaluation pushed more than one million requests across a cluster of ten cache servers, splitting the traffic evenly between configurations with Tiered Cache enabled and disabled. This separation allowed researchers to isolate local cache behavior from inter-tier network transfers. Although the test corpus utilized specific text assets that compressed at the expected 2.8x ratio, engineers acknowledge that a broader, more diverse corpus must be analyzed before applying these exact compression ratios as a permanent, fleet-wide constant.
The success of the Cache Transcoding prototype highlights the ongoing potential for deep architectural efficiencies within large-scale web infrastructure. By strategically balancing a minor increase in CPU overhead against substantial reductions in disk storage requirements and backbone bandwidth consumption, the approach successfully extends the lifespan and utility of existing hardware investments. Future development phases will evaluate higher zstd compression levels, test a broader spectrum of content types and object sizes, and explore the possibility of passing compressed objects directly to downstream components capable of handling them without intermediate decoding.
Leave a Reply