The rapid evolution of artificial intelligence is once again forcing a fundamental re-evaluation of data center architecture. While the initial wave of the generative AI boom relied heavily on massive parallel processing via graphics processing units (GPUs) and tensor processing units (TPUs) to handle straightforward conversational queries, the emergence of agentic AI is shifting the burden onto a different class of hardware. According to Mark Lohmeyer, vice president and general manager for AI and computing infrastructure at Google Cloud, the unique demands of agentic AI are placing unprecedented requirements on data center CPUs and the broader underlying infrastructure stack.
In a recent discussion, Lohmeyer contrasted traditional conversational AI with the emerging paradigm of autonomous agents. Traditional chat applications operate on a relatively straightforward request-and-response model. A user submits a prompt, and the system relies primarily on rapid inference executed across specialized accelerators like GPUs or TPUs. However, the operational nature of artificial intelligence agents introduces an entirely different tier of complexity, stretching the capabilities of modern computing environments.
With agentic systems, a user’s high-level intent is rarely resolved in a single step. Instead, that intent is systematically broken down into a series of smaller subtasks. These subtasks are then assigned to a collaborative team of digital agents capable of executing complex workflows. Such workflows might involve interacting with external application programming interfaces (APIs), browsing live websites, or even writing, testing, and running executable code—operations that lean heavily on CPU performance rather than raw accelerator throughput. Furthermore, agentic interaction sessions are significantly longer-lived than standard chat queries. They can span days or even weeks, involving multiple human participants and requiring the underlying system to preserve context and state reliably over extended periods.
This operational profile creates unique architectural challenges for data center operators. Agentic workloads are characterized by unpredictable behavior, remaining largely idle for long stretches of time while waiting for user inputs or external API responses, only to surge into intense bursts of activity when a task resumes. This dynamic makes them distinctly different from traditional CPU workloads. They require stateful persistence over long durations during periods of dormancy, coupled with the ability to handle high-intensity compute bursts when active. Designing infrastructure that can handle these contrasting states efficiently, securely, and with high performance has become one of the primary hurdles facing the semiconductor and cloud computing industries.
Demand for this next-generation capability is already high and accelerating at a rapid pace. As enterprises look to deploy autonomous agents to drive productivity and automate complex business processes, cloud providers must figure out how to scale their infrastructure to support this growth cost-effectively.
New Workloads and the Demand for Locality
To understand the scope of the transition, Lohmeyer drew a distinction between the specific compute resources required to run the agent software itself—a novel workload necessitating entirely new infrastructure models—and peripheral tasks such as accessing websites and third-party APIs. While web browsing and API calls are established paradigms that previously interfaced directly with human users, agentic workloads will exert a profound impact on both elements.
Because software agents are vastly more demanding than human operators, and because enterprises can spin up hundreds or thousands of agents almost instantaneously in ways that human user bases simply cannot match, the volume of these peripheral tasks will grow exponentially. To ensure optimal performance, these agents, along with their associated context, applications, and data, ideally need to reside in close physical or logical proximity to one another to minimize latency.
Despite the theoretical benefits of absolute data locality, the modern enterprise IT landscape remains stubbornly distributed. The vast majority of organizations continue to rely on a hybrid mix of multiple public clouds and private cloud environments. Consequently, high-performance networking remains critically important, both within individual data centers and across distributed cloud facilities. Dynamic, stateful agents require a novel type of computing infrastructure capable of supporting their continuous interactions with existing enterprise applications and data repositories, all coordinated seamlessly through robust network fabrics. Google, Lohmeyer noted, is actively investing across all of these foundational elements.
As the industry adapts to these requirements, the optimum balance of CPU-to-GPU compute for agentic AI is still actively evolving alongside the workloads themselves. However, the proportional requirement for CPU compute is definitively increasing as the industry transitions from the initial heavy lifting of training massive foundation models to the operational phase of agentic inference. While both CPU and accelerator workloads are projected to experience rapid growth, the share of total compute allocated to CPUs is shifting from a relatively minor component to a substantial and meaningful portion of the overall architecture. This ratio is expected to continue shifting as agents expand into new operational use cases and diverse deployment environments.
Hardware-Software-Workload Co-Design
Addressing the challenges of agentic inference cannot be solved by hardware alone; it demands deep co-design across the entire infrastructure stack, encompassing CPUs, system memory, high-performance storage, advanced networking, and the orchestration software that binds them together. Google Cloud has long championed a philosophy of workload-optimized infrastructure, a design philosophy that Lohmeyer argues is now more critical than ever in the agentic era.
Because agentic workloads are inherently bursty and leave CPUs idle for prolonged periods, systems must be engineered to handle periods of dormancy intelligently. Rather than holding dedicated compute resources hostage while an agent waits for a user or an external response, agents can effectively be "hibernated." During hibernation, an agent’s active state is serialized and sent to high-performance local storage, freeing up the CPU cores to process tasks for other active agents. When the dormant agent is triggered to wake up, its state must be reloaded into memory with minimal latency.
Achieving this level of rapid state restoration requires sophisticated orchestration software coupled with extremely high-performance storage tightly attached to the server hardware. To manage density and security simultaneously, Google Cloud utilizes microVMs—essentially streamlined, lightweight versions of traditional virtual machines designed to establish secure operational boundaries when multiple agents are packed densely onto a single physical server. While a traditional server might house around one hundred standard virtual machines, the same hardware footprint can accommodate up to one thousand microVMs.
Security and governance represent paramount concerns for enterprise deployment of agentic systems. At the foundational virtualization layer, microVMs offer distinct advantages. Their stripped-down feature set not only improves execution performance by removing unnecessary overhead but also minimizes the potential attack surface compared to bloated legacy virtual machines. In parallel, Google is investing in higher-level architectural controls, such as specialized agent gateways designed to enforce rigorous egress and network-level security policies.
Looking ahead, the operational requirement to host thousands of microVMs concurrently for agent-driven tasks will directly influence hardware design parameters, including core density requirements, storage attachment topologies, and the sophistication of the orchestration software required to manage them. Successfully scaling these parameters will yield higher overall hardware utilization rates, driving down the unit cost per agent. This economic efficiency is vital, enabling enterprise customers to scale their automated operations profitably without sacrificing system performance.
CPU Choice and Platform Diversity
As the industry prepares for this architectural shift, Google emphasizes that there will be no single, universal CPU choice that optimizes every conceivable agentic use case. To accommodate diverse enterprise requirements, Google Cloud offers a heterogeneous mix of x86 processors from leading vendors such as Intel and AMD, alongside its own custom-designed, Arm-based Axion CPU.
Different agentic workloads impose wildly varying demands on the underlying silicon. A data-intensive agentic workflow might require an architecture featuring exceptionally high per-core performance paired with massive amounts of local solid-state storage. Conversely, an agentic application focused primarily on high-volume web-serving or API orchestration might prioritize raw core count and energy efficiency above all else. Providing customers with the architectural flexibility to map specific agentic workloads to the most appropriate CPU option within the cloud portfolio remains a strategic priority.
Customer interest in Google’s proprietary Axion processor has been strong, according to Lohmeyer, spanning a variety of performance- and efficiency-optimized cloud instances, alongside rising demand for bare-metal configurations among large-scale enterprise clients navigating the transition to the agentic era.
