Data center CPUs see second wind from agentic AI

The artificial intelligence (AI) industry is seeing a shifting balance between training models and deployment for inference and applications, including agentic AI. While much of the discourse has been about the significance of graphics hardware (GPUs), central processing units (CPUs) are emerging to the fore again.

GPUs excel at parallel processing for massive matrix calculations, which can reduce training times from months to weeks or less. On the other hand, CPUs handle sequential tasks, data preparation, and orchestration, the kind of activities that demand low-latency and single-core compute.

The initial generative AI wave largely focused on text and multimedia generation, where CPUs functioned as a “head node”, responsible primarily for passing prompts, handling input and output, and scheduling tasks for GPU acceleration.

Today, however, the shift towards AI agents — systems and software that plan, loop, parse state, query databases, call external APIs, and execute sandboxes — is changing that dynamic. Because they rely on sequential processing, they are redefining the role of CPUs and creating new business opportunities for x86 architecture-based suppliers Intel and Advanced Micro Devices (AMD), said executives and analysts.

“Pre-agentic era, CPUs were used as a head node doing pre/post processing around GPU-bound training or inferencing. This needed GPU:CPU ratios as high as 8:1,” Parv Sharma, a senior analyst with Counterpoint Research, told ETElectronicsWorld.

“The CPU’s role is expanding from feeding accelerators to coordinating and executing agentic workflows…this is driving demand for more CPUs. Hence, GPU:CPU ratios have reached 2:1 or even 1:1 for agentic workloads,” Sharma added.

Morgan Stanley, in a research note in June, said that agentic AI requires “approximately one million times more compute” than the original conversational model.

In recent quarters, top executives at US-based data center CPU makers have provided color on the impact of agentic AI, projecting healthy business growth in the coming years.

Nvidia has termed agentic AI and reinforcement learning as new growth opportunities for CPUs, with the company having a standalone $20-billion revenue visibility tied to these chips. This would make Nvidia the “world-leading CPU supplier”, its President & CEO Jensen Huang has earlier said.

Lip-bu Tan, CEO of Intel, said at the most recent quarterly earnings call, “As AI expands from training to inference and increasingly to agentic and multi-agent systems, general-purpose server CPU density continues to increase, and our core server CPU franchise is growing faster than ever.”

“Demand signals from our customers are driving increased confidence. We see tremendous opportunities to leverage our strong x86-based general-purpose computing franchise to build more purpose-built computing products for the AI era,” Tan said.

Lisa Su, the chair & CEO of AMD, also said that demand is increasing both for accelerators and high-performance CPUs to orchestrate AI workloads. “We are seeing both stronger near-term demand…,” she said previously.

Based on customer demand and an increase in CPU compute requirements enabled by agentic AI, AMD expects its server CPU total addressable market (TAM) to grow at over 35% annually, reaching more than $120 billion by 2030. “As inferencing scales, and you have more agents and agentic AI, they all require CPUs,” Su said.

Counterpoint has forecasted that the server CPU market may reach around ~$200 billion by 2030.

Evolving competition

Beyond x86, the silicon market’s competitive landscape is changing by the day.

Qualcomm is preparing to re-enter the data center market, Arm is expanding its Neoverse IP platforms for high-efficiency AI compute, and hyperscalers including Google Cloud (Axion), Amazon Web Services (Graviton), and Microsoft Azure (Cobalt) continue to scale custom silicon.

“Now the competition is four ways. Nvidia becomes a merchant CPU vendor, Arm and Qualcomm enter with their own silicon, hyperscalers scale Axion/Graviton/Cobalt, and x86 (Intel and AMD) defend a growing pie,” Counterpoint’s Sharma said.

According to the research firm, Arm is projected to capture a “disproportionate” share of incremental hyperscale/cloud growth due to power efficiency and vertical integration, whereas custom silicon may dominate volume, and the merchant segment would be smaller but grow faster beyond 2028.

“Arm is capturing the fastest-growing share of new deployments, on track to approach half of data center CPU revenue by 2031,” Sharma said.

On market trends, Nvidia chief Huang said that traditionally, CPUs were designed with multiple cores that could be rented to customers, but agents demand compute power to execute tasks quickly. “The economics of the past were dollars per core. The economics of AI in the future are tokens per dollar or dollars per token. What we need to do in the future is to generate tokens and process them as fast as possible.”

For hyperscalers, cloud service providers, and telecom operators, this shift implies that capital expenditure (capex) allocation will need to be far more balanced between accelerated GPU computing and CPU clusters.