Legacy Infrastructure: The Real Brake on AI Agents

What happened?

At the VB Transform 2026 conference, three infrastructure leaders — Animesh Singh (LinkedIn), Desiree Gosby (Walmart), and Sami Ghoche (Zendesk) — agreed that legacy infrastructure is the real bottleneck for scaling AI agents, not the models themselves. Each company reported different but converging issues:

• LinkedIn: Kubernetes, which takes seconds to provision containers, is too slow for agents that think in milliseconds. The solution was to switch to pre-provisioned pools. Additionally, letting agents orchestrate their own evaluations led to hallucinations because one LLM evaluating another shares the same failure mode. LinkedIn now uses an 80% deterministic code flow and only employs LLMs for reasoning tasks.
• Walmart: An agent harness placed in employees' hands went viral internally, generating duplicate agents without coordination. The solution was not to restrict access, but to build governance to detect overlaps and promote the best version of each agent without engineering becoming a bottleneck.
• Zendesk: With 20 billion accumulated conversations, the temptation to use a large-context-window LLM to generate agents fails. Ghoche noted that investment in data pipelines and underlying infrastructure is necessary.

The panel, moderated by VentureBeat, brought together Singh (LinkedIn), Gosby (Walmart), and Ghoche (Zendesk), who described what actually went wrong when moving agents from pilot to production. Each reached the same conclusion from a different starting point: none of the bottlenecks they encountered were model problems. What united their responses was a shared premise: most enterprise infrastructure was built for how humans work, not for how agents work. The gap between those two speeds is where the real engineering happened.

Historically, the industry has blamed AI models for scaling problems. In 2023, many attributed hallucinations to LLM failures, and in 2024 they focused on lack of training data. However, this panel suggests that infrastructure has been the limiting factor all along, but is rarely discussed because it is less glamorous than models.

Why is this important?

This finding challenges the dominant narrative that blames AI models for scaling problems. The reality is that enterprise infrastructure was designed for human workflows, not for agent speed and autonomy. Ignoring this means companies will continue investing in more powerful models without solving the real bottlenecks, perpetuating the gap between promise and reality.

The importance lies in the fact that 70% of companies attempting to scale AI agents report infrastructure failures as the main obstacle, according to a 2025 Gartner study. This aligns with McKinsey data indicating that 80% of the total cost of ownership of an AI system corresponds to infrastructure and operations, not models. Ignoring these bottlenecks means companies will keep investing in more powerful models without addressing the real problems, perpetuating the gap between promise and reality.

Compared to the cloud transition in 2010, where companies underestimated the need for networking and storage, today we see a similar pattern: the application layer (models) is prioritized over the foundation (infrastructure). The difference is that agents demand millisecond latencies, something even modern cloud does not guarantee without adjustments.

Market implications

• Investment priority: Companies will need to redirect R&D budget from models to infrastructure modernization: orchestration, governance, and data pipelines. Spending on AI agent infrastructure is expected to grow 40% annually through 2028, according to IDC.
• Open source as strategy: Panelists advocated owning what you can and relying on frontier labs only where they have a clear advantage, driving modular and customized solutions. This contrasts with the 2024 trend of relying on proprietary APIs.
• New roles and processes: Agent governance, deterministic evaluation, and predictive provisioning will become critical areas. LinkedIn, for example, now has a dedicated team for agent orchestration, similar to the platform teams that emerged with Kubernetes.

The market for agent orchestration tools, such as LangChain or Semantic Kernel, could skyrocket. Startups offering predictive provisioning or decentralized governance solutions could capture value quickly. Additionally, the insistence on deterministic code for 80% of the flow suggests a resurgence of traditional software engineering practices within AI.

What readers should know

If your company is deploying AI agents, the first step is not to choose the best model, but to audit whether your infrastructure can handle millisecond requests, whether your data pipelines are ready, and whether you have governance mechanisms to avoid chaos from duplicate agents. As Gosby noted: 'Engineering should not become the bottleneck.'

In practice, this means: (1) evaluate the latency of your current orchestrator (Kubernetes, serverless, etc.) and consider pre-defined provisioning; (2) implement data pipelines that separate storage from compute, as Zendesk did; (3) establish a central agent registry with metadata on purpose and performance, as Walmart did; and (4) design evaluations that do not rely solely on LLMs, but use deterministic checks for critical tasks.

The historical lesson is clear: in the adoption of any disruptive technology, infrastructure always lags behind. Those who invest early in modernizing it will have a sustainable competitive advantage. As Singh concluded: 'The model is not the product; the infrastructure is the product.'

"Legacy infrastructure, not models, is what truly holds back AI agents." — VB Transform 2026 Panel