Nvidia Research: The Real Hero in AI Agents Isn't the Model

In a shift that could redraw developer priorities in the field of AI agents, Nvidia has published new research that turns prevailing assumptions upside down. Instead of the usual focus on the power of the language model itself, the findings suggest that the software layer surrounding the model — known as the "harness" — is the most influential factor in the success of complex tasks that require a long chain of decisions.

What Is the Software "Harness"?

The harness is the software layer that surrounds the language model and transforms it from a mere engine that answers questions into an agent capable of acting independently. This harness includes a set of core elements, most notably:

• The tools the model can use to carry out tasks.
• Memory and context management to maintain the workflow sequence.
• The rules and feedback mechanisms that guide the agent's behavior.

Adel Al-Hallaq, VP of Products in Nvidia's AI unit, explains that many people treat the agent as though it were merely a programming interface for the model, when in reality it is far more than that: "It's the model, and the surrounding structure we call the harness, and the set of tools it uses, in addition to the runtime environment, skills, and libraries we grant it access to."

A Perfect 100% Result

The striking part of the research is that the researchers were able — through designing a custom harness that masters memory management and includes a "supervisor" component — to push the Claude Opus model to achieve a perfect score of 100% on the ARC-AGI-3 benchmark. This test is a set of two-dimensional games with no instructions, in which the model is asked to discover on its own how to play and win, just as a human would.

For comparison, the same model without this advanced harness scored only 30%, even though that was the highest result among all tested models. OpenAI's models, meanwhile, initially scored less than 10%, prompting the company to conduct its own research, which showed that adjusting just two settings in the harness tripled its results — yet it still did not come close to a perfect score.

The Role of the "Supervisor" Agent

The most important secret lies in adding a supervisor component that works alongside the main agent tasked with performing the work. Al-Hallaq describes this component as behaving "much like an executive" who steers the agent when it strays off course, or begins exploring a dead end, or retraces a path it has already tried without benefit.

Although the idea of a supervisor agent is not entirely new, most users today rely on only a single layer in their harnesses, such as Claude Code or Codex. That is why Nvidia's researchers developed an enhanced harness they named "Agentic Variation Operators."

Why Do Long-Horizon Tasks Matter?

Long-horizon tasks are those that require linking multiple decisions together, and may stretch over days until the work is completed — unlike simply giving a quick answer to a question. The ability of agents to perform these tasks without losing focus or drifting from their goal is among the hardest challenges in this field.

Previous research has revealed the scale of the problem; a study conducted by Microsoft tested 19 models on document-editing tasks and found that all of them, including advanced models, filled the documents with errors. In fact, models operating autonomously were observed deleting users' files or entire databases, or resorting to unethical behaviors to achieve their goals.

The Harness Affects Cost Too

The harness's impact is not limited to performance; it extends to cost as well. In earlier research by Databricks, it was found that choosing the wrong harness could double the running cost of the same model. As Ali Ghodsi, the company's CEO, puts it: "You might choose the same model with a different harness and see the cost rise significantly. So the harness alone is enough to double what you pay."

A Call for an Open Ecosystem

The broader message Nvidia wants to convey is that open harnesses, like open models, give users a degree of control greater than they realize. Under the "Nemo" brand, the company offers a set of open tools for building harnesses — some commercial and many available for free. Al-Hallaq stresses that having an open agent ecosystem, one that allows control over the harness, infrastructure, and runtime environment, is what's needed to advance the ecosystem safely.

✦ بقلم فريق دروب أيديا