What Is an AI Harness?
By Scott Rippey
Over the course of about a month, I saw several people worth listening to each sit down to explain what an AI harness is. Not hype accounts. People who build these kinds of systems for a living, one way or another, plus one of the major cloud platforms.
They do not fully agree with each other.
One will tell you the harness is everything after the model, and that whatever interface you are looking at is specifically not part of it. Another says the harness is the whole experience, interface included, and then builds a custom interface as part of her harness.
Same word. Same month. Two answers that do not line up.
That is a stranger problem than hype, and a more annoying one. When somebody is wrong, you correct them. When a room full of careful people are each mostly right, and each describing something a little different, you are the one who has to work out where they are all standing.
So that is the job here. Not who wins. There is a core they all agree on, and an edge where they genuinely split, for real reasons. Once you can see which is which, the next harness take you scroll past gets a lot easier to place.
First: which harness?
Some of the confusion is older than agents. "Harness" was already a word in software long before any of this, and it has since been attached to several different layers. Those layers are not competing definitions, though. They are an ancestor, some parts, and the same thing viewed from further back.
| Term | What it means | How it relates |
|---|---|---|
| Test harness | A scaffold that feeds inputs to code and checks the outputs. Predates AI entirely. | The ancestor |
| Eval harness | Runs a model or agent against test cases and scores the behavior | One layer inside |
| Tool harness | Lets the model call functions, APIs, shells, browsers, databases | One layer inside |
| Context harness | Decides what the model sees, and when | One layer inside |
| Agent harness | The entire non-model system wrapped around a working agent | The thing people are arguing about |
| Production harness | The same system described wider: governance, cost control, observability, deployment | The same thing, further back |
So the plural collapses. There is really one contested idea here, and the older meanings are either where the word came from or a slice of the new one. When people argue about harnesses today, they are arguing about the agent harness. For the rest of this, "harness" means that.
Start with the model on its own
This part gets skipped a lot, and skipping it is why the rest stays fuzzy.
You send a model a prompt. You get text back. That is the whole transaction. It cannot open a file, run a test, remember you from yesterday, or check whether what it just told you was true. It is a very good text engine sitting in a box.
Which makes this the cleanest definition of an agent I have come across, from Simon Willison:
An agent is a model with tools, running in a loop, to accomplish a goal.
Now count the parts in that sentence. Tools. A loop. And a goal, which means something has to decide when it actually got there. Only one of those is the model. The rest is stuff somebody built and wrapped around it.
That wrapper is the harness:
Agent = Model + Harness
And that same little equation keeps turning up in places that did not coordinate with each other: a cloud platform's explainer, Martin Fowler's engineering write-up, a solo creator's walkthrough, a research post about agents rewriting their own scaffolding. LangChain puts it about as bluntly as it can be put: if you are not the model, you are the harness. When people who define a word differently keep landing on the same formula, there is usually something real underneath it.
It is also why the same model performs so differently depending on where you meet it. Claude in a chat window, Claude in a coding agent, and Claude inside a custom internal tool can be the identical model with wildly different results. The model is not what changed.
The core everyone agrees on
At the center, every source names the same three pieces.
- Tools, so it can reach outside the box. Read a file, run a command, hit an API, search the web.
- A loop, so it goes more than once. Do something, look at what came back, go again.
- A check that decides when it is finished. A test that passes, a rule that clears, a human who approves.
If you read The Agentic Spectrum, this is the rung underneath it. That piece sorts systems by how much they get to decide for themselves. This one is about what you build around a model to get there. A prompt is one instruction. An agent is a model looping on its own. The harness is what that loop is made out of.
That is the settled part. Nobody is fighting about any of it. They start splitting at the edges.
Where they actually split
It mostly comes down to one question: where does the harness stop?
| The question | One answer | Another answer |
|---|---|---|
| Is the interface part of it? | No. The harness is everything after the model, and it sits lower than the interface. The same harness runs behind a terminal, a chat window, or a web app with nothing underneath changing. Often there is no front end at all. | Yes. The harness is the whole experience, including the human experience. Build yourself a custom interface and it counts. |
| Is harness engineering the phase after context engineering? | Yes, in sequence. Prompt engineering, then context engineering, then harness engineering, each taking over from the last. | No, different axis. Context engineering asks what the model should see. Harness engineering asks what the system should prevent, measure, and correct. Both run at once. |
| How big is it? | Everything that is not the model. | Just the quality-control apparatus around the agent: the guides that steer it and the sensors that catch it. |
| Is it a thing you run, or a thing you tune? | A thing you run. | A search space you optimize, all the way up to agents rewriting their own harness. |
| Is a folder of instruction files a harness? | Yes, under the broad definitions. It is the instruction layer. | Not on its own under the tight ones. No loop, no tool execution, no stop condition. |
Both columns of that first row are defensible, depending on what you are doing. If you are building the engine, keeping the interface out of it is clean, and you get to reuse the engine. If you are shipping something a person sits in front of every day, the interface is doing part of the work, and calling it separate is splitting hairs.
The second row is the one place I will push back a little. Framing this as three eras in time is a useful story, but I do not think the era part holds up. The major labs describe it as a different question rather than the next step on a timeline, and one framing goes further: harnesses are the delivery mechanism for context engineering. Nested, not a relay race.
And then there is the loosest version, which usually shows up in walkthroughs covering AI more generally. You get shown a folder of markdown files, instructions, skills, and commands, and that gets called the harness. Under a wide definition, fair enough. That genuinely is the instruction layer. But if it is the only version you ever see, you would walk away picturing a harness as a few files, and miss the loop, the tool execution, the permissions, and the thing that decides when to stop.
Nobody here is lying to you. They are standing at different altitudes, and hardly any of them say which one. That missing scope label is the entire source of the confusion.
What fills the space between the tight version and the broad one
It is worth seeing what actually lives in that gap, because the gap is most of the engineering. Take the tight definition (tools, loop, stop condition), keep widening, and this is what you pick up along the way.
| Layer | What it decides |
|---|---|
| Instructions | The standing rules that apply before anyone types anything |
| Context assembly | What enters the working context, and what stays out |
| Tools | What it can reach, and how hard each one is to misuse |
| Execution loop | How it acts over time, and how many attempts it gets |
| State and persistence | What survives when the context window does not |
| Sandbox | What the environment physically allows |
| Permissions | Which actions need a human first |
| Verification | How anyone knows the work is actually good |
| Observability | Whether you can reconstruct what happened afterward |
Two of those deserve a note, because they are where harnesses stop being demos.
Verification splits in two. Some checks are computational: tests, typecheck, lint, schema validation, a citation that either exists or does not. Fast, deterministic, cheap, and not up for debate. Others are inferential: a review agent, a faithfulness judge, a critique pass. Slower, semantic, probabilistic. Good harnesses run the cheap deterministic ones first and save the expensive judgment for what genuinely needs it. Nobody should ask a language model whether the TypeScript compiles.
Steering happens before and after. Martin Fowler's framing is the useful one here: guides feed forward and sensors feed back. Guides shape the work before it happens, through rules, docs, examples, and constraints. Sensors catch what happened after, through tests, linters, logs, and reviewers. A harness with only guides is hoping. A harness with only sensors is rework.
A harness is a restraint, not an amplifier
Here is the part that got me once I lined all of these up next to each other.
Everybody reaches for the word harness as though it means more power. Bolt this on and your model can finally go do things. So go back and look at what these people actually built.
- A bug-triage harness with an investigate mode that will explain a root cause and structurally cannot write a single line of code. Its author also deliberately made her own data connector worse than the general-purpose one, so the agent pulls a few defined fields instead of wandering through every trace.
- Anthropic's write-up on long-running agents, which is mostly rules about what the agent is not allowed to do. Do not try to build the whole app at once. Do not mark a feature finished without testing it. Do not delete a test that is failing.
- Engineering advice that is almost entirely caps. Around ten tools rather than thirty. An instruction file under sixty lines. No new rule unless a specific failure earned it.
- A quality-control harness that is a set of checks whose whole job is to reject work.
- Even the loosest version, the folder of markdown files, whose sharpest component is a reviewer agent that catches unsourced claims and sends the work back.
Those are all restrictions.
And look at the word itself. A harness on a horse. A climbing harness. The harness in a car seat. Not one of those makes anything more powerful. They exist to keep something from going where it should not.
A harness is not the thing that lets your model do more. It is the thing that stops it from doing everything.
The reason that matters operationally: a model that can do anything is a model you cannot predict, and you cannot ship what you cannot predict. Fence off what it is allowed to touch, and that same model turns into something that does one job the same way every time.
A harness I built
Every time I push code, a security review fires before anything reaches the remote. I built it months before I had the word for it, and I never once called it a harness until the definition made it obvious.
| Layer | What it does |
|---|---|
| Context selection | Decides what gets looked at |
| Orchestration | Runs the same code past more than one model, from more than one company |
| Permissions | None of them get to change a single line. They only report. |
| Persistence | Every finding is written to a database and surfaced on a dashboard |
| Human gate | Nothing moves forward until I have worked the queue by hand |
Every piece of that is a limit. That is a harness.
That is one push. A coordinator hands the same code to seven specialists, two of them running models from a different company entirely, and their six findings collapse into one decision that still waits on me.
How to place any claim you read
When somebody tells you what a harness is, start from the assumption that they are right, then work out which layer they are standing on. A folder of instruction files? The tools and the loop? The checks that reject bad work? The whole product a person sits down in front of? Those are all real answers. They are answers to different questions, and once you can hear which question somebody is answering, the disagreement stops being confusing and starts being useful.
If you are the one building it, the question flips around. It is not what can I let this thing do. It is what should it not be able to do. What does it not get to touch, how does it know when it is finished, and what has to be true before the work counts as done.
Which is also why "which model should we use?" is rarely the first question worth asking. The model is the most interchangeable part of the system. The harness is where your process, your data, your rules, and your quality bar actually live.
The plain version
Strip out every bit of terminology and here is the whole thing:
A harness is everything you build around a model so it can do real work without you having to watch it. Tools so it can reach outside its box, a loop so it can try more than once, and a check that decides when it is done.
Some people mean a folder of instruction files by it, some mean the tools and the loop, some mean everything including the interface. They are each describing a real thing at a different size.
And none of it makes the model smarter. It is the same model either way. What changes is how much room you gave it, and how carefully you defined what done means.