Nvidia redefines CPU for AI: Vera bets on single-thread speed

What happened?

Nvidia has revealed details of its new Arm Vera CPU, describing it as a 'maximum single-thread performance CPU at scale,' in contrast to traditional designs that maximize core count. Vera features 88 Olympus cores with SMT support (176 threads), LPDDR5X memory with 1.2 TB/s bandwidth, and 3.4 TB/s of inter-core bandwidth, three times more than any existing data center CPU, according to the company.

In benchmarks leaked by Phoronix, Vera matched or surpassed AMD and Intel x86 in single-thread workloads, even 'crushing' them in some scenarios, according to Tom's Hardware. Nvidia claims it offers 1.8x better performance in AI inference compared to the competition, though these figures should be taken with caution until independent benchmarks are available. The company also previewed its next-generation Rigel cores (Arm v9.2) for the Rosa CPU, promising even higher per-core performance thanks to 'better instruction delivery,' more L2 cache, and improved memory handling.

This announcement comes in a context where Nvidia had already shown Vera in Linux benchmarks, and AMD responded with data from its 256-core EPYC Venice (Zen 6), claiming 3.3x better performance at the rack level (100 kW). However, the comparison is complex because Venice uses many more cores and a different manufacturing process. Nvidia counters that for sequential AI, single-thread performance is what really matters.

Why is it important?

Nvidia's approach challenges the conventional wisdom that more cores are always better. For agent AI and reasoning workloads, where each step depends on the previous one, single-thread speed is critical. Vera is optimized for sequential inference, not massive parallel processing, which could give it an edge in applications like chatbots, virtual assistants, and autonomous reasoning systems. Nvidia describes these workloads as 'sequential by nature': a reasoning model executes one step, then the next, and so on until it gets the answer; no parallelism speeds up that process.

Moreover, Vera is part of Nvidia's strategy to integrate CPU, GPU, and networking into a cohesive ecosystem (like the Grace Hopper superchip), directly competing with AMD and Intel x86 processors in the data center. The company aims to deliver predictable performance and low latency, not just a high core count. This echoes the era when Intel dominated with efficient single-thread architectures, before AMD bet on many cores with Zen.

Market implications

If Vera delivers on its promises, it could redefine CPU architectures for AI, pressuring AMD and Intel to also optimize single-thread performance. AMD has already responded with its 256-core EPYC Venice, claiming 3.3x better rack performance, but the comparison is complex: Venice uses many more cores and a 3nm process, while Vera has 88 cores and is on an unknown node (likely TSMC 4nm or 5nm). For users, Vera could mean faster inference and lower latency in AI applications, though adoption will depend on integration with the CUDA ecosystem and Nvidia's software. Companies like Microsoft, Google, and Meta, which already use Nvidia hardware for AI, could benefit if Vera is integrated into their data centers.

However, Vera's success is not guaranteed. The data center CPU market is dominated by x86, and transitioning to Arm requires recompiling software and optimizing workloads. Nvidia already has experience with Arm in its Grace superchips, but mass adoption will take time. Meanwhile, Intel is developing its own AI CPUs (like Granite Rapids) and AMD is strengthening its EPYC line. Competition will intensify in 2025-2026, when Vera's first deployments are expected.

Nvidia is not simply adding cores, but aiming to deliver predictable performance and low latency for sequential AI.

What readers should know

Vera is not a general-purpose CPU; it is specifically designed for AI and agent workloads. Its single-thread performance is key, but direct comparisons with x86 are complex due to differences in benchmarks and configurations. The next-generation Rigel promises further improvements, but there are no concrete release dates yet. For companies investing in AI, Vera represents a viable alternative to traditional CPUs, provided the software is optimized for its Arm architecture. Nvidia has created a new product category: 'maximum single-thread performance CPU at scale,' which could be a turning point if the industry adopts this philosophy. However, analysts recommend waiting for independent benchmarks and real-world use cases before making purchasing decisions. Time will tell if Vera is the next big breakthrough or just a footnote in data center history.