Writer cuts token costs by 40% by optimizing the AI harness

The problem: 'tokenmaxxing' as an ROI crisis

The enterprise AI industry faces a paradox: as more computing power is thrown at foundation models, costs skyrocket in production. According to a new paper from researchers at Writer, the root of the problem is tokenmaxxing: a trend where developers use massive contexts and brute-force token consumption as a substitute for good system design. This phenomenon, first described on The Pragmatic Engineer blog, has become normalized because "it's the cheapest solution at the moment," according to Waseem AlShikh, CTO and co-founder of Writer, in a statement to VentureBeat. The typical approach involves generating, executing, failing, reinserting the error and more context into the window, and retrying. While it often works for coding tasks, it has become the default reflex for any agentic workload.

“Your bill is tokens-per-task multiplied by price-per-token, and most teams only look at the second number,” AlShikh explains. “In agentic workloads, tokens-per-task accumulate — each loop iteration retransmits the growing context — and it accumulates faster than prices drop.”

Tokenmaxxing leads to business failures like routing simple tasks to premium models, using the LLM as a lazy search index, or building unconstrained agentic loops that spiral out of control. Since output tokens cost much more than input tokens — typically 3-4 times more in models like GPT-4 or Claude — inefficiency becomes a silent budget killer. A 2024 study by A16Z already warned that inference costs can surpass training costs in production. The Writer paper quantifies this impact: in complex tasks like financial report generation, agentic loops can consume up to 10 times more tokens than necessary.

The solution: optimize the harness, not the model

The Writer paper tackles the problem from the orchestration layer that wraps the foundation model, called the AI harness. By optimizing this layer — which is entirely under the developer's control and requires no fine-tuning of the model — they achieved dramatic reductions in tokens per task and a drop in cost per successful task of up to 61%, while maintaining consistent quality. Existing techniques like prompt compression, budgeted reasoning, concise coding, or speculative decoding treat the model in isolation and do not address the inefficiency of agentic loops. Writer's proposal directly attacks workflow design, an approach reminiscent of data pipeline optimizations in the big data era, but applied to generative AI.

The concept of a "harness" is not new: in software engineering, it refers to the integration layer that connects components. Writer adapts it to AI systems, including task routing, context management, loop control, and tool selection. By optimizing these elements, unnecessary retransmission of context is avoided and retry cycles are limited. The results show that it is possible to cut token consumption by up to 40% without losing accuracy in complex tasks, such as extracting structured data from legal documents.

How does harness optimization work?

The study breaks down the components of the harness and proposes specific techniques:

• Task routing: Classify requests by complexity and direct simple ones to cheaper models (e.g., a small 7B parameter model) and complex ones to premium models. This avoids wasting GPT-4 on trivial tasks.
• Context management: Implement dynamic context windows that only retain relevant information, instead of forwarding the entire history. This drastically reduces input tokens.
• Loop control: Set retry limits and early stopping criteria. For example, if the model reaches a high-confidence response, the loop stops.
• Tool selection: Choose the right tool for each subtask, avoiding unnecessary calls to expensive APIs.

These optimizations are purely engineering-based and do not require changing the underlying model. Any team can apply them immediately, democratizing efficiency without relying on model providers. In comparison, techniques like model distillation or fine-tuning require significant resources and access to training data.

Impact and consequences

This research comes at a critical time when companies are desperately seeking to reduce AI costs without sacrificing performance. According to a 2025 Gartner report, 60% of generative AI implementations in production exceed their planned budget. The tokenmaxxing trend has become normalized because "it's the cheapest solution at the moment," according to AlShikh, but the paper demonstrates that a more efficient approach is possible.

The market implications are clear: model providers will have to compete not only on price per token but on total system efficiency. Companies like OpenAI and Anthropic already offer cheaper models (GPT-4o mini, Claude Haiku), but harness optimization can further reduce costs. Startups building agentic applications, such as workflow automation, will be able to offer more competitive pricing. And engineering teams will need to rethink their architectures to avoid token waste.

In AlShikh's words: “Price cuts become an anesthetic. They mask the fact that the loop itself is bleeding.” Writer's solution attacks the hemorrhage directly. Additionally, the paper suggests that these optimizations could be integrated into orchestration platforms like LangChain or LlamaIndex, expanding their impact.

However, there are limitations. The study focuses on specific Writer tasks, such as report generation and data extraction. Generalization to other domains requires validation. Moreover, harness optimization can increase system complexity and require ongoing maintenance. Nonetheless, the results are promising and offer a roadmap for the industry.

In summary, the Writer paper not only exposes the tokenmaxxing crisis but provides concrete tools to overcome it. In an environment where efficiency is key to the economic viability of enterprise AI, optimizing the harness is emerging as an essential practice.