The Co-Failure Ceiling: Why Using Multiple AI Models Doesn't Always Improve Results

What happened?

A team of researchers published a study on arXiv (preprint 2606.27288) evaluating 67 frontier models from 21 providers — including GPT-4o, Claude 3.5 Sonnet, Gemini 2.0, and Llama 3.1 — and discovered a phenomenon dubbed the co-failure ceiling. The common premise that combining models with low error correlations creates a safety net is mathematically incorrect: the real limit is not how often models disagree, but the percentage of queries where all fail simultaneously. According to the study, companies using multiple models underestimate failure rates by 2.25 times. That is, if a set of models has a co-failure ceiling of 20%, the actual error rate of the orchestrated system will be at least 20%, regardless of how low the pairwise correlation is.

Why is it important?

Orchestration architectures — routers, cascades, and mixture of agents (MoA) — add a 'shadow price' in latency, maintenance complexity, and governance risks. For example, a router like the one recently presented by VentureBeat (a 1.5B model achieving 93% accuracy) introduces additional latency and infrastructure costs. If the co-failure ceiling is not considered, companies invest in expensive infrastructure for performance gains that do not exist. Lead author Josef Chen warns that naive majority voting among unequal models can worsen performance: weak models cluster together and outvote the strongest one. This is analogous to what happened in early recommendation systems, where combining weak algorithms diluted the accuracy of the best one.

The market impact is significant: companies like OpenAI, Anthropic, and Google compete to be the single dominant model, but the study suggests that multi-model orchestration may not be the panacea many hoped for. Orchestration startups like Portkey, Helicone, and LangChain could see their value proposition questioned if clients realize the benefits are marginal. Additionally, the operational cost of maintaining multiple APIs and managing governance (compliance, privacy, billing) adds to latency. According to industry data, an orchestrated query can take 2 to 5 times longer than a single-model query.

Consequences and recommendations

The study offers clear guidelines: only combine models within the same quality band. If quality cannot be matched, it is better to use a single high-end model. However, there is a positive exception: in MoA architectures, a diverse set of models with low correlation outperforms 'Self-MoA' (querying the same model multiple times) when quality is matched. This has implications for designing systems like those used in the Chatbot Arena benchmark, where aggregating votes from multiple models has shown improvements, but only if the models are of similar quality.

For developers, the main lesson is that the pairwise error correlation metric should not be used to predict the system's absolute accuracy. Instead, the co-failure ceiling of the model set should be directly calculated — a free test that determines whether orchestration will actually pay off. This can be done by taking a subset of validation prompts and seeing how many fail all models simultaneously. If that number is high, orchestration will not help.

“Companies pay the cost of orchestration upfront (latency, complexity, multi-provider operations) assuming a diversity dividend will come later. But if the co-failure ceiling is high, that dividend never arrives.” — Josef Chen

Compared to past events, this echoes the debate on ensembles in classical machine learning: for years it was assumed that combining models always improved performance, until it was shown to work only if models are diverse and of similar quality. The co-failure ceiling is the LLM version of that lesson.

What should readers know?

• Do not assume more models are always better: diversity without quality control can be detrimental. The study shows that in some cases, a single high-end model (like GPT-4o) outperforms combinations of weaker models.
• Measure the co-failure ceiling before implementing orchestration. It is a simple calculation that avoids unnecessary investments.
• Prioritize a single high-quality model over a combination of mediocre ones. This is especially relevant for startups with limited resources.
• In MoA, diversity with matched quality is superior to repeated self-querying. For example, combining Claude 3.5 Sonnet, Gemini 2.0, and Llama 3.1 (all similar performance) yields better results than querying GPT-4o three times.
• The co-failure ceiling also applies to majority voting systems: if 51% of models fail on a query, majority voting will give the wrong answer. This is critical in high-risk applications like medical diagnosis or algorithmic trading.

In conclusion, the study by Chen and his team is a wake-up call for the industry. Multi-model orchestration is not a silver bullet; it requires rigorous analysis of candidate models. Companies that ignore the co-failure ceiling will continue to pay hidden costs without achieving expected benefits, while those that calculate it can optimize their systems efficiently.