Cloudflare has announced a major architectural overhaul of its DNS propagation pipeline, dramatically improving performance for its global network. The company revealed that a new targeted build planner and unified scheduler have cut peak memory usage by 85% and halved the median time it takes for a DNS change to reach the edge. The improvements address long-standing scaling challenges as customer zones grow increasingly massive, with some individual domains now holding more than 20 million DNS records and projections pointing toward hundreds of millions.
When Cloudflare last detailed its DNS propagation pipeline in 2022, its Zone Builder service processed roughly 250 DNS record changes every second. Today, that volume has nearly tripled to about 700 changes per second, alongside approximately 100 other background events, such as zone setting modifications. However, engineers quickly realized that raw event volume was not the primary bottleneck. Instead, the core issue lay in the nature of the work required by certain events and how computational demands scale with the size of a zone.
Every modification—whether a record update or a configuration tweak—triggers a "build" process. This mechanism reads the current database state, translates it into the edge-compatible format required by Cloudflare’s authoritative DNS servers, and writes the resulting data to Quicksilver, the company’s distributed configuration store. In the past, handling zone-wide settings often necessitated full builds, which required reading, encoding, and comparing every single record while holding the entire zone in memory. While economical for small domains, this approach became unsustainable for massive zones containing tens of millions of records that exceed available memory limits.
The challenge in designing a more efficient system centered around accurately determining the scope of work. While a record-level event explicitly identifies what has changed, zone-level events can originate from a multitude of internal Cloudflare products. These events frequently lack explicit context detailing why a zone was modified and can alter shared data across multiple zones that share the same domain name. Engineers noted that allocating too much processing capacity wastes resources, while allocating too little risks leaving stale or incorrect DNS data exposed at the edge.
To solve this, Cloudflare built a targeted build planner capable of determining the smallest possible build that can be mathematically proven safe, paired with a unified scheduler that intelligently combines concurrent work for the same zone. Rather than forcing a full rebuild of a zone with tens of millions of records whenever a minor configuration setting changes, the system now selectively updates only the affected components. This targeted approach dramatically reduces computational waste.
The transition also involved refining how events communicate changes. The event producer was updated to explicitly include modified zone fields and settings, complete with historical and new values where available. By tracing each setting through the code responsible for generating DNS outputs, engineers were able to classify precisely what elements each setting could affect. If an event lacks sufficient detail or only provides a generic timestamp indicating that something changed, the system conservatively falls back to a full build, ensuring that uncertainty never compromises data integrity.

Quicksilver relies on two distinct types of data keys for its operations: record keys, which house the specific DNS answers for a given zone and name, and metadata keys, which instruct authoritative DNS servers on how to utilize a zone, including the location of wildcard entries and nameservers. By distinguishing between these components, the planner can precisely execute targeted builds ranging from resource-only updates to comprehensive metadata refreshes, ensuring that every output key remains byte-for-byte identical to what a traditional full build would produce.
The architectural improvements extend to how the system handles complex naming scenarios. During customer onboarding, offboarding, and other lifecycle transitions, multiple internal Cloudflare zone objects can temporarily exist for a single public domain. Quicksilver manages this by storing a shared metadata key that acts as a directory, informing authoritative servers which internal zone IDs are authorized to answer queries. The updated pipeline ensures that when nameserver or wildcard changes occur, both the specific records and the shared metadata key are updated accurately, while localized record changes leave the overarching metadata untouched.
Before execution, the system performs a fresh validation check against read-only database replicas to account for any state changes that occurred while an event was queued. The DNS Kafka Aggregator introduces a deliberate delay to accommodate normal database replication lag, and a final verification step rejects reads if a replica has fallen behind, preventing stale data from propagating.
The introduction of a unified scheduler has further streamlined operations by merging record-level tasks and full-zone workflows into a single framework. When multiple events accumulate for a single zone before a build commences, the scheduler consolidates the required work. To protect system resources from being monopolized by a single busy domain, a per-zone rebuild interval ensures that rapid-fire changes are naturally batched together, while safeguards prevent failing builds from entering infinite retry loops.
The impact of these changes on infrastructure performance has been substantial. Measurements over a 24-hour period revealed that nearly 93% of all builds successfully avoided rebuilding complete zones. Full builds dropped by approximately 40%, and those that do still occur are typically triggered by events like first-time zone publications that naturally involve small, newly created domains.
Aggregate working-set memory across the deployment fell from an average of 49 GiB to 7 GiB, representing an 86% reduction, while peak memory usage dropped by 85% from 79 GiB to 12 GiB. Furthermore, the efficiency gains allowed engineers to safely shorten internal pipeline buffers and rebuild intervals, cutting the median build completion delay in half from roughly 7.7 seconds to 3.3 seconds, and reducing extreme tail latency spikes twelvefold.
Through rigorous side-by-side testing using mirror environments and automated comparison tools, Cloudflare confirmed that targeted builds achieve parity with legacy full builds while drastically reducing system overhead. While full builds will remain necessary for specific scenarios such as initial zone publications or untracked changes, the revamped pipeline ensures that Cloudflare’s DNS infrastructure can effortlessly scale to meet the demands of the internet’s largest domains.