The hidden cost of letting agents run
The economics of AI coding agents change once they stop answering single questions and start working on their own. A one-shot prediction is cheap. A long, unsupervised run is not: reasoning steps pile onto tool calls, which pile onto feedback loops, and the token bill grows with every cycle. Nvidia researchers have published a paper describing a system that attacks that bill from an unexpected angle, not by making the underlying model cheaper to run, but by rewriting the scaffolding that surrounds it.
According to the researchers, their method, called SoL-Pi, reduces token consumption by almost half while leaving performance roughly unchanged. The work was reported by The Decoder, which notes that the system targets the control layer known as the harness rather than the model itself.
What the harness actually does
The harness is the layer of software sitting between a model and the environment it operates in. It shows the agent what state it is in, decides how actions get executed, and determines what feedback returns to the model afterward. Familiar products such as Codex, Claude Code, and OpenClaw rely on harnesses of this kind to turn a language model into something that can edit files, run tests, and iterate.
Because that layer governs so much of the loop, it has an outsized effect on cost. Every redundant tool call, every oversized context window, every verification step that fires too late adds tokens that were not strictly necessary. Until now, most efficiency work has concentrated elsewhere, on reducing the price of a single token through faster attention kernels and serving infrastructure, on compressing models via quantization, or on routing work to smaller, cheaper models.
Why harness optimization is hard
Tuning the harness by hand is difficult because its responsibilities are tightly interwoven. Tool usage, context management, verification routines, and abort logic all interact. A change that trims tokens in one stage can introduce failures somewhere else, or simply shift the cost into a later phase of the run where it is harder to see.
In practice, engineers read through long execution traces, spot recurring failure patterns, and translate those observations into code changes. It is slow, manual work that depends on someone noticing the same mistake for the tenth time.
How SoL-Pi automates the search
SoL-Pi takes that human loop and automates it. A research agent observes the traces produced by another agent, proposes modifications to the harness, and then tests those proposals inside prepared environments. Candidate changes are screened through both capability checks and efficiency checks, and only the variants that hold performance steady while cutting cost are kept.
The authors describe the approach as drawing on recursive self-improvement: one AI system is effectively improving the operating conditions of another. A key detail is that the evaluation used to judge the final result happens only after the harness has been frozen, and that held-out evaluation does not feed back into the search. That separation matters, because it limits how much the optimization process can tune itself to the tests it will eventually face.
The scale of the experiment
The search was broad. Across 535 executable environments, SoL-Pi explored 152 distinct directions. The task mix included:
- 495 tasks built from GitHub issue and pull request pairs
- 40 synthetic test cases designed for the evaluation
- More than 3,000 total runs
- Over 60,000 agent-environment interactions
The researchers present that volume as evidence of how widely the system searched, but they also caution that more searching does not automatically produce better outcomes. Breadth increases the chance of finding a useful harness configuration, yet it does not guarantee one.
The measured savings
The headline numbers are substantial. On EdgeBench, SoL-Pi saved 50 percent of tokens compared with Codex and 54.3 percent compared with Claude Code, according to the researchers, while performance stayed approximately level. The paper frames the overall token reduction as up to 49 percent with little change in task capability.
That framing is important. Cutting token usage is easy if an agent is allowed to fail more often, skip verification, or abort early on legitimate tasks. The claim here is different: the savings come from leaner control logic rather than from degraded behavior, which is why the capability checks sit alongside the efficiency checks in the selection process.
The overfitting question
There is a known hazard in this line of research. Earlier work has shown that automatically optimized harnesses tend to overfit, becoming highly efficient on the traces they were tuned against while generalizing poorly to new situations. The Nvidia authors appear to be aware of this, which is why the final assessment is conducted on a frozen harness and isolated from the search loop.
Even so, the caveat is worth carrying forward. The environments used here, while numerous, are still a finite sample of the messy reality that coding agents encounter. Whether a harness refined across 535 environments holds up against a genuinely unfamiliar repository, a flaky test suite, or an ambiguous bug report remains an open question that only wider deployment will answer.
What it means for agent economics
If the results hold outside the lab, the implications are practical rather than philosophical. Teams running autonomous coding agents pay per token, and the long unsupervised sessions that make agents useful are precisely the ones that make them expensive. Squeezing nearly half the tokens out of a workflow without losing capability changes how many of those sessions a budget can sustain.
The work also suggests a shift in where optimization effort is worth spending. Model-level efficiency has been the default target for years, and it certainly still matters. But the harness is where decisions about context, verification, and retries get made, and as this paper illustrates, those decisions are where a large share of the waste accumulates. The gap between the model and the environment may be the most productive place left to look.
This article is based on reporting by The Decoder. Read the original article.
Originally published on the-decoder.com
