Harness engineering: the model plus everything around it

The most useful equation to come out of the agent literature this year is also the shortest: agent equals model plus harness. Birgitta Böckeler set it down on Martin Fowler's site in April 2026, after OpenAI's Ryan Lopopolo had used the phrase "harness engineering" in February to describe how the Codex team works, and the term spread quickly because it names something practitioners already knew. The model is rented and changes under you every few months. The harness is everything you build around it: the instructions, the tools it can call, the context it is given, the memory it keeps, the checks that run on its output, and the controls that stop it doing harm. The harness is where reliability lives, and it is the part you own.

I find the framing clarifying for enterprise work because it separates the two questions clients tend to run together. "Is the model good enough?" is a question about a component you did not build. "Is the system reliable?" is a question about the harness, and it is answerable with engineering.

Close

A model at the centre, surrounded by the parts of a harness: instructions, tools, context and retrieval, memory, evaluation and tests, guards and permissions, observability. Arrows show the loop: the model proposes, the harness checks, the result feeds back.

What a harness is made of

Strip away the product names and a harness has a handful of parts.

• Instructions. The standing guidance the model receives: the system prompt, the AGENTS.md or CLAUDE.md file, the operating rules. This is the part most teams over-invest in, as the evidence below suggests.
• Tools. What the agent can actually do: query a system, call an API, run a test, write a file. The set of tools, and what each is permitted to do, defines the agent's reach.
• Context and retrieval. What the agent is shown about the current situation, and how that is selected. This is where the ontology work I wrote about separately plugs in.
• Memory. What persists between runs: state, decisions, learned preferences, and how it is kept from growing without bound.
• Evaluation. The tests and benchmarks that say whether a change to any of the above made the agent better or worse.
• Guards and permissions. The executable checks that stop an action before it happens: a hook that refuses a destructive command, a policy that routes a decision above a threshold to a person.
• Observability. Logs of what the agent actually did, so that drift is seen in production rather than inferred from evaluations.

The agent loop that ties these together is simple: the model proposes, the harness checks, the result goes back to the model, until the checks pass or a limit is hit.

What the evidence says

Most writing about harnesses is advice. Francois Zaninotto's survey, The State of AI Harness Engineering 2026, is unusual in collecting the measurements, and the numbers are worth reading before any of the advice. I take the following from it; the figures are his, and a caution comes with them. They are drawn from separate experiments of different sizes, almost all on coding agents, several from vendors reporting on their own products. Read them as indicative of where the gains are, not as settled industry-wide results, and follow his links to the original studies before quoting any of them in a decision.

The harness matters as much as the model. The same model run through eight different harnesses succeeded between 68 and 88 per cent of the time on the same tasks. A twenty-point spread from the wrapper alone is the whole argument for the discipline.

Fewer tools can be better. Vercel removed 80 per cent of the tools from an agent; success went from 80 to 100 per cent, tokens halved, and latency fell from 724 seconds to 141. Every tool is a decision the model has to make, and most of them were noise.

Written rules are weak controls. Across the repositories surveyed, only 4 to 16 per cent of the rules written in instruction files are actually enforced by anything. A rule in prose is a hope. A hook that refuses the action is a control. Permission rules written in advance blocked 20 per cent fewer bad actions than per-action approval.

Generated context can hurt. A study at ETH Zurich found that machine-generated context files reduced task success compared with giving the agent nothing, while raising cost by a fifth or more. The lesson is not that context is bad; it is that context has to be curated by someone who knows what matters.

Most harnesses are untested. Sixty per cent of the harnesses surveyed contained neither tests nor evaluations. That is the same finding as in every other branch of software, and it is why I treat an agent without an evaluation set as a prototype regardless of what it is called.

More agents is not more reliability. Adding a reviewer agent reduced success by 8 per cent in the scenarios tested, and chains with more than four agent-to-agent handovers almost always failed. The pattern that works is a small number of specialised agents with narrow tools, not a committee.

Harness value decays. Rules written to compensate for a model's weakness become dead weight when the next model does not have the weakness. Zaninotto's recommendation, which I have adopted, is that every control in a harness should carry the incident that justified it and the condition under which it can be removed.

From coding agents to enterprise agents

Almost all of this evidence comes from coding agents, where the task is bounded, the tests are executable and the cost of a wrong action is a failed build. Enterprise agents are different in three ways, and the harness has to carry the difference.

The checks are not all executable. A coding harness can run the test suite. An agent that drafts a supplier response cannot run the supplier. The evaluation layer has to include human review of a sample, scored against a written rubric, and the sampling has to be driven by the agent's own confidence, which is where calibrated decision models like the one I wrote about in the Jev article become useful.

The guards are policy, not syntax. The hook that matters is not "do not delete files" but "do not commit the organisation to a price", "do not send to a customer without review", "do not act on a record whose owner is not identified". Those guards have to be written against the ontology, because they refer to business entities and states, and they have to be enforced in code, because a rule in the prompt is a hope.

The audit trail is a deliverable. In a coding harness the log is for debugging. In an enterprise harness the log is what the compliance function reads. Observability has to record what the agent saw, what it decided, what checks ran, and who approved what, in a form a person who was not there can follow.

Where the current frameworks fall short

This is my judgement from building agentic systems and the evaluation harnesses around them in enterprise settings, and from teaching engineers to do the same. It is not a benchmark result.

They are designed for the demo, not the second month. Every framework makes it easy to give an agent many tools and a long prompt and hard to take tools away. The evidence above says the opposite is what works. A harness should start empty and earn each tool.

They assume the check is executable. The coding-agent lineage shows: the loop expects a test suite to pass. Enterprise work needs a loop that can end in "a named person must look at this", with the case, the agent's confidence and the reason recorded, and most frameworks treat that as an exception path rather than a first-class outcome.

Permissions live in the prompt. The frameworks let you write "never do X" in the instructions and call it a control. The survey's finding that a small fraction of written rules are enforced by anything matches what I see. Permissions belong in the tool layer, where the call is refused, and the frameworks that make this easy are the exception.

Nothing expires. Rules accumulate. No framework I have used tracks why a rule exists or when it should be removed, so harnesses grow heavier as models grow more capable, and the compensations for last year's model become this year's latency.

Observability is for developers, not for the people accountable. The traces are rich and the compliance function cannot read them. The audit log an enterprise needs is a different artefact from the debug trace, and it has to be designed.

What I would want from a framework, and what I build around the ones I use: a harness that starts with nothing, a check that can end in a human, permissions enforced below the model, a reason and an expiry on every rule, and a log written for the person who will be asked about it.

How I build one

In a forward-deployed engagement the harness is most of the work, and I build it in a fixed order.

1. Write the evaluation set first. Thirty to a hundred real cases with the correct outcome recorded, before any prompt is written. Everything after is measured against it.
2. Define the entities and actions. The small ontology: what the agent can see, what it can do, and the states in which each action is allowed. This becomes both the retrieval guide and the guard list.
3. Give it the fewest tools that work. Start with two or three. Add one only when a case in the evaluation set fails for want of it.
4. Enforce every rule that can be checked mechanically in code, and route the rest to a person. If a rule is important enough to write and a machine can test it, it is important enough to check automatically. Rules that need judgement, whether a draft commits the organisation to something, whether a case is sensitive, become review steps with a named reviewer, not prose in a prompt.
5. Put human review where confidence is low, not everywhere. Route by calibrated threshold, record the decision, and feed the reviewer's correction back into the evaluation set.
6. Log everything the agent did, in a form the client's own people will read, and watch the production logs rather than the benchmark once it is live.
7. Date every control. Each guard carries the case that justified it and the condition for removing it, so the harness shrinks as the model improves rather than accreting forever.

The handover at the end of the engagement is the harness and the evaluation set, with a named owner for each. The model is a configuration value. That is the practical meaning of "agent equals model plus harness": the client keeps the harness, and the model is whatever is best this quarter.

Sources

• Ryan Lopopolo, OpenAI, the "Harness engineering" post on working with Codex in an agent-first world, February 2026.
• Birgitta Böckeler, on martinfowler.com, April 2026, for the "agent = model + harness" formulation.
• Francois Zaninotto, The State of AI Harness Engineering 2026, Marmelab, September 2026, for the survey figures quoted above.
• From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents, arXiv, July 2026.