Cloudflare has announced a major engineering achievement involving its core DNS routing and resolution engine, known internally as "Big Pineapple." The platform, which powers high-visibility services including the widely used 1.1.1.1 public resolver, Gateway DNS, DNS Firewall, and the AS112 project, manages upwards of 250 billion DNS cache entries concurrently across its global server fleet. At such an immense operational scale, even the most minuscule memory inefficiencies can compound into massive hardware requirements. Recognizing that wasting a single byte per cache entry translates to more than 250 gigabytes of wasted memory fleet-wide, engineers embarked on a rigorous optimization campaign.
Through five successive modifications to how cache entries are structured and stored in memory, Cloudflare successfully reduced the per-entry footprint by more than 50 percent. Across the entire global infrastructure, these targeted changes freed approximately 100 terabytes of RAM—an amount roughly equivalent to the total memory capacity found across 130 of the company’s modern Gen 13 servers. Crucially, the engineering team achieved these remarkable space savings without compromising execution speed. In fact, cache insert throughput increased by 43 percent while lookup latency dropped by 19 percent, proving that memory conservation and performance enhancement can go hand in hand when dealing with low-level data structures.
What Big Pineapple Caches
When a server running Big Pineapple boots up, it starts with an entirely empty cache. As incoming DNS queries stream in from users around the globe, the cache progressively fills until it reaches its maximum capacity threshold. Once full, the system relies on eviction policies to remove older or less frequently requested items to accommodate incoming data. The physical size of the cache varies dynamically depending on the specific data center location and its local traffic patterns.
A significant factor influencing memory consumption is the use of EDNS Client Subnet, commonly referred to as ECS. When ECS is enabled, authoritative nameservers often return distinct answers tailored to the specific network neighborhood of the requesting client. Consequently, the caching layer must store multiple distinct versions of what is conceptually the same query. This behavior multiplies the total number of entries and increases the memory footprint required for each one, rendering optimizations especially critical in geographical regions with heavy ECS utilization.
Every item residing in the cache represents a key-value pair. The key defines the exact parameters of the query, incorporating the domain name, record type, authentication status, and unique tagging data. Meanwhile, the value payload contains the actual DNS response—comprising the answer, authority, and additional record sections—alongside important metadata such as creation timestamps, hit counters, and Time-to-Live values. Both the key and value structures presented opportunities for deep architectural refinement, particularly because standard programming language abstractions often carry hidden overhead that becomes unnecessary once data is safely persisted in memory.
Benchmarking Memory Usage
To accurately evaluate the real-world impact of each adjustment, the engineering team established a reliable benchmarking environment. They filled the test cache with syntactically randomized entries mirroring production traffic distributions: roughly 56 percent standard A records, 25 percent AAAA records for IPv6, and 19 percent TXT records, with each entry containing between one and four individual records. TXT records served as a generalized proxy for all non-standard record types, featuring randomized sizes varying between 64 and 224 bytes to closely approximate variable-length responses observed in real-world operations.
Monitoring memory usage at this granular level required the implementation of a custom memory allocator wrapping the standard system allocator in Rust. This tool tracked the precise count and size of allocations generated by every single cache entry. Alongside memory metrics, the team measured insert throughput and lookup latency to ensure that memory optimizations did not inadvertently introduce bottlenecks. Because test inputs only approximate production environments—with actual process memory depending heavily on shifting traffic mixes, cache occupancy rates, and allocator states—the team also verified their findings by tracking resident set size across live production instances throughout the phased rollout.
The Cost of Capacity and Structural Bloat
A major initial focus centered on how dynamic collections handle capacity. Standard growable vectors in Rust store three fundamental components: a pointer pointing to heap-allocated data, the current length of the collection, and the total allocated capacity. When an item is pushed into a vector, the underlying logic checks whether the length exceeds capacity, triggering a reallocation if necessary. However, once a DNS response is fully processed and stored inside the cache, it remains entirely static and is never modified again.
Under these immutable conditions, the capacity tracking field serves no functional purpose while continuing to consume eight bytes of memory per vector. Furthermore, any over-allocated heap space reserved for future growth remains permanently unused. By transitioning these fields to fixed-size slices and specialized string types that eliminate capacity tracking entirely, the engineering team reclaimed significant memory overhead. Across a baseline of 250 billion cache entries, eliminating unnecessary capacity fields and excess heap reservations saved more than 15 terabytes of RAM.
Fewer Lists and Improved Memory Locality
Further architectural improvements targeted how response sections are organized. Instead of maintaining separate, independent lists for the answer, authority, and additional sections—each carrying its own pointers and length indicators—the team consolidated the data into a single contiguous list. Using compact two-byte offsets to designate the start of each section replaced the heavier pointer-and-length pairs required by individual collections. This structural compression eliminated redundant pointers and reduced the base entry size significantly.
These structural updates also triggered positive secondary effects related to memory alignment. In systems programming, compilers automatically insert padding bytes to satisfy hardware alignment constraints, rounding total struct sizes up to designated multiples. Removing small fields often eliminated surrounding padding entirely, causing structs to shrink by amounts exceeding the raw size of the deleted fields. Complementing this, packing multiple boolean flags into a single bitflag representation minimized padding further, yielding unexpected compound space savings.
Pruning Record Owners
Another major optimization involved the handling of record ownership. Every DNS record possesses an owner field identifying the specific domain name associated with it. In a vast majority of queries, this owner matches the queried domain name identically. For instance, a standard query for an A record on a domain returns records sharing the exact same owner string. On the network wire, the DNS protocol handles repeating domain names efficiently using compression pointers defined in RFC 1035, referencing the original occurrence rather than repeating the full string.
While wire compression works well for transmission, storing full owner strings alongside every individual record in the cache previously traded away memory for speed to avoid expensive parsing overhead on the hot path. However, recognizing that most records share an identical owner with the queried domain, engineers modified the schema to omit the owner field entirely for matching records, inferring it dynamically at read time from the cache key. When record owners diverge—such as when a CNAME points to an external resource—the system falls back to storing the full name explicitly. This adjustment eliminated heap allocations for the vast majority of cached records.
Refining Enum Sizing and Record Layouts
Handling varied DNS record types presented another structural challenge. In systems programming languages utilizing algebraic data types, enums always occupy a memory footprint equal to their largest possible variant. In the initial design, the enum holding record data was sized around the largest DNS record type present, which required 136 bytes and resulted in a total enum size of 144 bytes once alignment padding was factored in.
Because lightweight record types like IPv4 addresses require only four bytes and IPv6 addresses require sixteen bytes—and together make up over 80 percent of total traffic—the vast majority of records were wasting more than 120 bytes of padding. To resolve this inefficiency, engineers boxed the larger, less common enum variants, shifting their bulk to separate heap allocations while allowing common variants to reside inline.
While boxing successfully reduced the inline size of frequent records, it introduced potential drawbacks, including allocator bin quantization waste and degraded memory locality due to scattered heap pointers. To eliminate these residual costs, the team pivoted to storing record data directly as raw byte buffers. Rather than maintaining a list of parsed enum variants, records were encoded into a contiguous byte slice prefixed with simple length indicators.
This raw byte representation removed per-variant enum overhead and avoided scattered heap allocations while significantly improving CPU cache locality. Although records could no longer be accessed via random indexing—requiring sequential iteration instead—the small number of records per entry rendered the traversal cost negligible. Furthermore, when constructing outgoing DNS responses, many common record types could be copied directly from the cached byte buffer without re-serialization, accelerating the lookup path and reducing CPU overhead.
Production Results and Future Outlook
The cumulative impact of these five engineering optimizations transformed the efficiency of Cloudflare’s DNS infrastructure. Benchmark measurements confirmed that the per-entry memory footprint dropped from 953 bytes down to 420 bytes, representing a 56 percent reduction. Simultaneously, per-entry allocations fell from 1.1 kilobytes to 461 bytes.
In production environments, these improvements manifested as dramatic drops in resident memory across all monitored percentiles. At the 99th percentile, instance memory consumption declined from 9.3 gigabytes to 5.3 gigabytes, marking a 43 percent reduction. At the 90th percentile, memory usage fell from 6.5 gigabytes to 3.8 gigabytes. Across the entirety of Cloudflare’s global fleet, the combined efforts liberated roughly 100 terabytes of working-set memory.
Performance metrics validated the design choices. Cache insert throughput climbed by 43 percent, jumping from 625,000 entries per second to 893,000 entries per second. Meanwhile, cache lookup latency improved by 19 percent, decreasing from 828 nanoseconds to 670 nanoseconds. Cloudflare intends to reinvest the newly freed memory capacity into expanding cache sizes globally, which will improve hit rates and lower upstream query volumes without increasing overall infrastructure footprints.
Leave a Reply