Nvidia accelerates in CPUs for AI: Grace and Vera in agentic data centers

What happened?

Nvidia, the company that has dominated the GPU market for artificial intelligence, is accelerating its foray into the CPU market for data centers. In a recent interview with Tom's Hardware, Ian Buck, vice president of high-performance computing and creator of CUDA, revealed that the company has shipped “hundreds of thousands of standalone Grace servers”. This figure surpasses previous numbers: in May, 2.5 million Grace CPUs were reported in total, and in February, a collaboration with Meta was announced to deploy standalone Grace servers. Buck's comments suggest the deployment scale is even larger, indicating significant early adoption by customers.

Additionally, Nvidia introduced Vera, a CPU specifically designed for agentic data centers, where AI agents require a tighter balance between CPU and GPU. While traditional AI workloads used up to eight GPUs per CPU, agentic workloads tend toward a 1:1 ratio, demanding more powerful and efficient CPUs. Vera represents Nvidia's response to this new demand, with up to 2x performance-per-watt improvements in some CPU workloads, according to the company.

Why is it important?

This move marks a seismic shift in Nvidia's strategy. The company, which has dominated the GPU market for AI, now aims to compete directly with Intel and AMD in the data center CPU segment. Historically, Nvidia has relied on third-party CPUs (such as AMD EPYC or Intel Xeon) in its DGX and HGX systems. However, with Grace and Vera, the company is betting on an integrated CPU-GPU ecosystem that promises deep optimizations for AI.

The evolution of AI workloads, especially autonomous agents, is redefining hardware requirements. According to Buck, AI agents require more balanced processing between CPU and GPU, as reasoning, planning, and code execution tasks often fall on the CPU. This contrasts with large model training, where the GPU is the bottleneck. Moreover, Nvidia's recent ~$1 trillion loss in market capitalization, as investors turn to CPU makers like Intel, underscores the urgency to diversify. The CPU foray could help Nvidia capture a broader market and reduce its dependence on GPU sales for AI.

What consequences will it have?

The success of Grace and Vera could alter the balance of power in the data center chip market. If Nvidia manages to impose its CPUs, it could erode Intel and AMD's market share while offering integrated CPU-GPU solutions optimized for AI. However, the challenge is enormous: Intel and AMD have decades of experience and established ecosystems, with CPUs dominating from enterprise servers to supercomputing. Nvidia must also convince customers that its CPUs offer the same performance and efficiency as established alternatives, especially in non-AI workloads.

A key factor is the software ecosystem. Nvidia has CUDA, widely used in AI and high-performance computing. If the company can extend CUDA to fully leverage its CPUs, it could offer a significant competitive advantage. However, Intel and AMD are not standing still: Intel is pushing its Xeon CPUs with integrated AI accelerators, while AMD bets on its Zen architecture and the ROCm platform. The battle will be fought on both hardware and software fronts.

For users and businesses, this means more options and potentially better prices. Competition could lead to faster innovation and lower costs for AI servers. However, there is also a risk of ecosystem fragmentation, especially if Nvidia opts for proprietary solutions that limit interoperability.

What should readers know?

• Nvidia has already shipped hundreds of thousands of standalone Grace servers, indicating significant early adoption, with Meta as one of the first major customers.
• The new Vera CPU is designed for agentic data centers, where the CPU-GPU ratio approaches 1:1, and promises up to 2x performance-per-watt improvements in CPU workloads.
• This shift reflects Nvidia's adaptation to a market where AI agents are changing computing patterns, moving away from massive training toward real-time inference and reasoning.
• Competition with Intel and AMD will intensify, but Nvidia starts with the advantage of its CUDA ecosystem and AI expertise. However, Intel and AMD have a massive installed base and established relationships with data centers.
• Investors and businesses should monitor how CPU versus GPU demand evolves in the coming years. If agentic workloads become widespread, demand for balanced CPUs could surge, benefiting Nvidia if it positions itself well.
• Nvidia's success in CPUs will depend on its ability to deliver competitive performance in traditional workloads, not just AI. Otherwise, customers may opt for hybrid solutions with Intel/AMD CPUs and Nvidia GPUs.

In summary, Nvidia is taking a bold step into the CPU market, a move that could redefine the semiconductor industry. With Grace and Vera, the company is not only seeking to diversify its offerings but also to anticipate future AI needs. Time will tell if this bet pays off, but what is clear is that the war for the data center has just intensified.