Cloudflare has announced a major memory-reduction milestone achieved across its core DNS infrastructure, freeing up roughly 100 terabytes of RAM by radically shrinking the footprint of its global cache. The optimization effort targeted "Big Pineapple," the underlying platform powering several high-profile Cloudflare services including the widely used 1.1.1.1 public resolver, Gateway DNS, DNS Firewall, and the AS112 project. Operating at a scale that routinely maintains over 250 billion DNS cache entries simultaneously, Cloudflare engineers found that wasting even a single byte per entry resulted in a massive aggregate penalty across their server fleet.
By implementing five successive and sophisticated changes to how cache entries are structured and stored in memory, the engineering team slashed the per-entry footprint by more than 50 percent. Crucially, these memory savings were not achieved at the expense of performance. Instead, the architectural adjustments resulted in a 43 percent increase in cache insert throughput and a 19 percent reduction in lookup latency. According to the company, the total memory reclaimed across the fleet is equivalent to the amount of RAM found in roughly 130 of Cloudflare’s high-density Gen 13 servers.
What we cache
On a cold start, Big Pineapple boots up with a completely empty cache. As incoming DNS queries stream in from users around the world, the cache gradually fills up until it reaches its maximum capacity threshold. Once that ceiling is met, older or less frequently requested items are systematically evicted to accommodate new entries. The precise physical size of the cache varies dynamically depending on the specific data center and local traffic patterns.
When features like EDNS Client Subnet (ECS) are actively utilized, authoritative name servers alter their responses based on the geographic location and network of the end user. Consequently, the platform must cache multiple distinct versions of the exact same query to ensure accurate routing. This behavior significantly increases both the total number of entries and the amount of memory consumed by each individual record, making optimization initiatives especially critical for locations handling high volumes of ECS traffic.
Every single item housed within the cache is structured as a key-value pair. The key is responsible for identifying the exact query parameters, incorporating the requested domain name, record type, authentication status, and unique tags. Meanwhile, the value section contains the complete DNS response payload—spanning the answer, authority, and additional record sections—alongside essential operational metadata such as creation timestamps, hit counters, and Time-to-Live (TTL) values. Examining these data structures revealed numerous opportunities to eliminate structural overhead left behind by standard programming abstractions.
Benchmarking memory usage
To accurately quantify the real-world impact of each proposed modification, the engineering team established a rigorous benchmarking environment. They artificially saturated the cache with randomly generated entries meticulously calibrated to mirror the traffic distribution observed in live production environments. This benchmark traffic mix consisted of 56 percent standard A records, 25 percent AAAA records for IPv6 traffic, and 19 percent TXT records, with each entry containing between one and four individual records.
To stand in for all non-A and non-AAAA record types, TXT records were assigned randomized sizes ranging between 64 and 224 bytes, closely matching the average response footprints observed for variable-length DNS records in the wild. Memory consumption was monitored using a custom memory allocator wrapping Rust’s standard system allocator, capturing precise metrics on the number and size of allocations tied to every single cache entry.
Alongside memory tracking, the benchmarks measured insert throughput and lookup latency to ensure efficiency gains did not introduce processing bottlenecks. Because synthetic benchmarks only approximate real-world conditions, the team also continuously measured resident memory usage across live production instances throughout the staged rollout to verify actual fleet-wide savings.
The cost of capacity
A foundational challenge addressed during the optimization project involved the behavior of standard vector types in Rust. A typical dynamic array stores a pointer to heap-allocated data, its current length, and its total allocated capacity. When an item is added, the vector checks whether the length exceeds capacity, triggering a reallocation if necessary.
However, once a DNS response is successfully written to the cache, it remains completely static and is never modified again. This rendered the capacity tracking field entirely obsolete, yet it continued to extract an ongoing tax of 8 bytes per vector. Furthermore, any over-allocated heap space reserved by the vector for future growth remained permanently wasted.
By replacing standard dynamic vectors and strings with fixed-size alternatives that drop the capacity field entirely, the team eliminated unnecessary heap reservations. Across a cache containing over 250 billion entries, replacing these types saved substantial amounts of memory while entirely removing excess heap fragmentation.
Fewer lists, fewer pointers
Rather than isolating the answer, authority, and additional record sections into separate lists—each carrying its own pointer and length overhead—the engineers restructured the data to reside within a single unified list. By utilizing compact numeric offsets to mark the beginning of each section, they replaced bulky pointer-and-length pairs with lightweight 16-bit integer offsets.
This structural consolidation removed multiple separate lists, saving dozens of bytes per entry. Furthermore, the removal of small fields triggered secondary structural benefits by altering memory alignment requirements. Rust automatically inserts padding bytes into data structures to satisfy hardware alignment rules, meaning the removal of small fields frequently eliminated surrounding padding and caused data structures to shrink well beyond the raw byte count of the removed fields alone. For instance, packing multiple boolean status flags into a single bitflag further reduced structural padding.
Dropping the owner
Every individual DNS record traditionally carries an owner field indicating the domain name to which the record belongs. In the vast majority of standard queries, this owner is identical to the domain name specified in the original query key. For example, a standard query for an A record on a apex domain returns responses whose owners match the queried name identically.
However, when records involve canonical name redirection or other complex routing scenarios, record owners can diverge from the initial query domain. While traditional DNS wire formats utilize name compression pointers to handle repeated domain names efficiently over the network, maintaining full string ownership locally for every cached record proved memory-intensive. Reconstructing compressed pointers during high-speed cache lookups introduced unacceptable CPU overhead on the critical path.
To resolve this, the engineering team redesigned record structures to omit the owner field entirely for the vast majority of records where the owner matches the query domain. Instead, the system dynamically infers the owner at read time by referencing the cache key already present during every lookup, entirely bypassing the need for duplicate heap allocations. For exceptional cases where the record owner genuinely differs from the query domain, the system falls back to storing an explicit pointer to the full name on the heap.
Enum sizing and boxing the variants
Rust enums function as algebraic data types, meaning an enum’s memory footprint is always dictated by its single largest variant, regardless of which variant is actively stored. For DNS records, storing diverse types within a single monolithic enum created massive inefficiencies. While standard IPv4 address records require only a few bytes and IPv6 addresses require slightly more, some specialized record types involved extensive multi-field structures.
Because the largest legacy record variant demanded significant space, the entire enum—including variant tags and alignment padding—inflated dramatically. Given that high-frequency record types accounted for the vast majority of all incoming traffic, the vast majority of cached records were wasting substantial amounts of memory on idle padding space.
To mitigate this imbalance, engineers experimented with boxing the larger and less frequent enum variants, moving their bulky payloads out of the inline structure and onto the heap behind an 8-byte pointer. While this successfully reduced the baseline size of high-frequency records, it introduced secondary complications. Heavy reliance on boxing increased allocator overhead, as memory allocators round allocations up to fixed size classes, and scattered record data across disjointed regions of the heap, degrading CPU cache locality.
Storing records in wire format
Seeking to eliminate the performance penalties associated with boxed variants and individual record parsing, the team ultimately shifted toward a hybrid approach. Instead of storing fully parsed programming language enums or attempting to cache complete wire-format DNS messages—which would complicate handling conditional features like DNSSEC flags—they elected to store record data as raw bytes accompanied by lightweight structural metadata.
Under this optimized architecture, records are stored within a contiguous byte buffer prefixed by short length indicators rather than parsed as individual heap objects. This layout packs the data tightly together, drastically improving CPU cache locality and allowing high-frequency record types to be copied directly into outgoing responses without costly field-by-field serialization work. Only complex records containing domain names requiring name compression undergo targeted parsing during the response generation phase.
To construct these optimized byte buffers efficiently during cache insertions, the system utilizes a reusable scratchspace buffer that persists across operations. Because the buffer has already grown from previous write cycles, it rarely requires fresh reallocations, effectively replacing a multitude of small allocations with a single streamlined memory copy operation.
The results
Production telemetry collected following the rollout demonstrated a profound impact on whole-process resident memory utilization across the global server fleet. As successive software releases deployed to production instances, memory usage dropped steadily across all statistical percentiles. At the 99th percentile, typical per-instance memory consumption plummeted from 9.3 gigabytes down to 5.3 gigabytes, representing a 43 percent reduction in resident memory. At the 90th percentile, memory usage dropped from 6.5 gigabytes to 3.8 gigabytes.
Across the entire benchmarking suite, the combined effect of the five optimizations drove the net per-entry memory footprint down from 953 bytes to just 420 bytes, marking a 56 percent decrease. Simultaneously, total per-entry memory allocations dropped from 1.1 kilobytes to 461 bytes. Operational performance metrics improved concurrently, with cache insert throughput jumping by 43 percent and lookup latency dropping by 19 percent.
Cloudflare indicated that the substantial volume of memory reclaimed through these architectural refinements will be reinvested directly into expanding cache capacity, allowing the platform to serve a higher volume of requests locally, improve overall cache hit rates, and reduce upstream query traffic without expanding its hardware footprint.
Leave a Reply