Harnesses and brokers

What the layer covers

Everything between a model deciding something and the world changing happens here. Reading the model's proposed action, checking whether it is permitted, executing it, capturing the result, deciding whether the task is done, and recording enough that a person can reconstruct the sequence later.

This is where most of the engineering effort in an agentic system actually goes, and it is the layer most often underestimated. Building something that calls a model and runs a tool takes an afternoon. Building something that does it repeatedly, under load, with real permissions, without losing track of state, and in a way anyone can debug afterwards takes considerably longer.

It is also the layer where the openness test bites hardest. Data formats and model APIs have converged enough to be reasonably portable. Harnesses have not. A great deal of what a team builds here is harness-specific unless they deliberately push definitions out into portable forms, which is what the standards layer exists to enable.

Harness and broker are different jobs

These two terms get used interchangeably and should not be. Keeping them apart clarifies a great deal about how larger systems are put together.

A harness runs an agent. It owns the loop, holds the conversation and task state, executes tools, enforces limits, and produces a result. It is where the work happens.

A broker decides which harness, agent, or model-powered tool should receive a piece of work, hands it over, and collects the result. It does not run the loop. It routes, applies policy about who may do what, and gives the rest of the system one place to send requests.

Small systems have one harness and no broker, and that is correct. The broker becomes worth its complexity when there are several execution options, when different work has different policy requirements, or when calling systems should not need to know which agent implementation exists this month.

The agent loop, honestly described

Stripped of vocabulary, an agent loop is short:

1. Assemble context: instructions, available tools, relevant facts, history, and the current request.
2. Ask the model what to do next.
3. If the model proposes a tool call, check whether it is allowed, run it, and capture the result.
4. Add the result to the context and repeat.
5. Stop when the model produces a final answer, a limit is reached, or a policy check fails.

The difficulty is not in the shape. It is in every branch that the shape hides. What happens when a tool times out. What happens when the model proposes the same failing call four times. What happens when a tool returns two megabytes of output. What happens when a user cancels mid-step, or when the process restarts while a task is in flight. What happens when a tool result contains text that looks like an instruction.

That last one deserves emphasis. Anything a tool returns is data, not instruction. Web pages, file contents, issue descriptions, and email bodies can all contain text addressed to the agent. A harness that treats retrieved content as authoritative is exploitable by anyone who can put text where the agent will read it. Keeping that boundary is a harness responsibility, and it cannot be delegated to the model.

What a harness is responsible for

1. Context assembly

Deciding what the model sees on each step. This includes trimming history, selecting relevant facts, supplying tool definitions, and ordering content so that stable material comes first. Most quality problems that look like model problems are context assembly problems.

2. Tool execution

Actually running the call, with timeouts, argument validation, output size limits, and errors returned in a form the model can act on. A tool error that crashes the loop is a harness defect. A tool error reported back as a result the agent can respond to is correct behavior.

3. Authority enforcement

Deciding whether a proposed action is permitted at all, before it runs, based on rules the agent cannot change. This is covered in its own section below because it is the responsibility most often skipped.

4. State management

Keeping track of the conversation, the task, intermediate results, and progress, in a way that survives restarts if the work is long-running.

5. Limits

Bounding steps, wall-clock time, token spend, and repeated failures. Without limits, a confused agent will loop until something else stops it, and something else is usually a bill.

6. Recording

Writing down what happened: which model, which prompt, which tool, which arguments, which result, which decision. This is what makes the auditable property real, and it has to be built in rather than added later, because you cannot record the past.

7. Termination

Deciding when the work is finished, when it has failed, and when it needs a person. Ending cleanly is underrated. Agents that cannot recognize failure keep trying, and each attempt costs money and sometimes does damage.

Tools are the surface area

An agent can only do what its tools let it do. That makes tool design the highest-leverage work in this layer, and it is closer to API design than to prompt writing.

A few properties separate tools that work from tools that produce confusing behavior.

Named for intent, not for implementation

A tool called get_open_orders_for_account is chosen correctly far more often than one calledrun_query, because the model is selecting from names and descriptions. Broad, generic tools push the difficulty onto the model at every call. Specific tools encode the decision once, in code, where it can be tested.

Descriptions written for the caller

The description is not documentation for humans who will never read it. It is the entire basis on which a model decides whether this tool applies. It should say what the tool does, when to use it, when not to, and what the arguments mean in domain terms.

Results shaped for reasoning, not for display

Returning a raw API response with sixty fields wastes context and buries the answer. Returning the six fields that matter, with units and a note about what was filtered out, gives the model something it can actually use and keeps the loop affordable.

Errors that suggest the next move

An error saying the account identifier was not found, and that identifiers look like a certain pattern, lets an agent correct itself. A stack trace does not. Because the model reads errors as input, error text is part of the interface.

Few enough to choose between

Tool selection quality degrades as the list grows. Beyond roughly a couple of dozen, models start choosing badly and the fix is not a better model, it is fewer tools in scope for a given task. Scoping tools per task rather than exposing everything is one of the most effective quality improvements available.

Authority and approval

The most consequential design decision in this layer is where the boundary between what an agent may propose and what it may do is drawn, and how that boundary is enforced.

There is a common mistake worth naming. Teams write the boundary into the system prompt: do not delete anything, do not send email without asking, only touch these tables. This is not enforcement. It is a request. It fails when the model misreads intent, when a task is unusual, and reliably when a document the agent reads contains text designed to override it. Instructions in a prompt cannot constrain a system that is capable of ignoring them.

Real authority is enforced outside the model, by the code that executes the tool call:

• Capability scoping. The agent is only given tools it should have. A tool that is not registered cannot be called.
• Argument constraints. The tool itself limits what it will accept: these tables, this directory, this recipient domain.
• Credential scoping. The credentials the agent operates with cannot perform the forbidden action, so a mistake fails at the boundary rather than succeeding.
• Approval gates. Certain classes of action pause and wait for a person, with the proposed action shown clearly enough to judge.
• Reversibility preference. Where a reversible form of an action exists, prefer it. Draft rather than send. Branch rather than push. Soft delete rather than hard.

A useful test: if the model were replaced with one that behaved adversarially, what could it actually do? Whatever the answer is, that is your real security boundary. Everything else is a preference.

State, memory, and context

Three different things get called memory, and separating them prevents a lot of confusion.

Working state is the current task: the steps taken, the results so far, the goal. It lives for the duration of the task and should be durable enough to survive a restart if the task is long.

Conversation history is what was said. It grows without bound and has to be trimmed, summarized, or selectively recalled, because sending all of it on every step is expensive and lowers quality.

Long-term memory is what should persist across tasks: preferences, prior decisions, learned facts about a codebase or a customer. This is the hardest of the three to do well, because writing to it indiscriminately produces a store full of noise that degrades every future retrieval.

The discipline that helps most is deciding explicitly what gets written to long-term memory and requiring a reason. Memory that accumulates by default becomes a liability. Memory that accumulates on purpose becomes the thing that makes an agent feel useful over weeks rather than minutes.

When more than one agent helps

Splitting work across several agents is often reached for too early. It adds coordination cost, more failure modes, and more places for context to be lost. It is worth doing when there is a specific reason, and the reasons are narrower than the enthusiasm suggests.

Good reasons:

• Different authority. One part of the work needs write access to production and the rest does not. Separating them keeps the wide-permission surface small.
• Genuinely parallel subtasks. Five independent files to review, or five sources to check, with no dependency between them. Wall-clock time drops in proportion.
• Context that will not fit. A task spanning more material than one context can hold, split so each worker holds a slice and a coordinator holds only conclusions.
• Independent verification. One agent produces, another checks, and the checker is prompted to disagree rather than to confirm. This catches a class of confident errors that self-review does not.

Weak reasons: because the diagram looks better, because a task has several conceptual phases that one agent could handle sequentially, or because assigning personas feels like specialization. A single agent with good tools and clear limits beats a committee of agents passing summaries to each other in most real workloads.

When work is split, the coordination cost is real and should be designed rather than assumed. Each handoff loses context, so what crosses the boundary needs to be explicit: the task, the constraints, the relevant facts, and what to return. Handoffs that pass a free-text summary and hope tend to degrade quietly as tasks get harder.

Why a system ends up with several harnesses

Teams usually intend to standardize on one and usually do not, for reasons that are mostly good.

Coding work wants a harness with deep filesystem and repository awareness. Customer-facing work wants tight latency and conservative defaults. Data analysis work wants query tooling and result handling. Background automation wants durability and retries more than interactivity. These are genuinely different products, and a single harness that serves all of them well is rare.

Different harnesses also arrive at different times. A team adopts a coding agent, then a vendor tool ships with its own agent embedded, then someone builds an internal one for a workflow no product covers. The realistic goal is not to prevent this. It is to make sure the definitions that matter live outside any one of them.

That is the practical argument for the standards layer. If skills, tool connections, agent identity, and workflow shape are expressed in portable formats, then having three harnesses is an operational detail. If they are expressed in one harness's configuration language, then having three harnesses means maintaining the same definitions three times and watching them drift.

What brokering actually decides

When a broker is present, it answers a small number of questions that would otherwise be answered by whichever caller happened to be written first.

• Which executor. Based on the kind of work, the data involved, current availability, and cost.
• Whether at all. Some requests should be refused or escalated rather than routed, and that check belongs in one place.
• Under whose authority. The identity and permissions the work runs with, which are properties of the request rather than of the executor.
• With what budget. Step, time, and spend limits attached at dispatch rather than assumed by each harness.
• Where the record goes. One usage and outcome record across every executor, which is otherwise impossible to assemble.

The value is concentration. Without a broker, every one of these decisions is made implicitly, differently, in each calling application. With one, they are made once and can be changed without touching callers.

Debugging an agent

Agents fail differently from ordinary software. There is rarely a stack trace pointing at a line. There is a sequence of individually reasonable steps that added up to the wrong outcome, and the only way to find the problem is to read the sequence.

That makes the trace the primary debugging artifact, and it needs to contain more than most logging captures by default. A usable trace shows the exact context sent on each step rather than a template, every tool call with its full arguments, every result including errors, the model and version used, and the reason the loop stopped. Anything summarized before storage tends to omit precisely the detail that explains the failure.

Reading traces also reveals a pattern worth knowing: most bad outcomes trace back to a step much earlier than the visible mistake. A tool returned an empty result, the agent interpreted empty as none exist rather than as query was wrong, and every subsequent step built on that. Fixing the final step does nothing. Fixing the tool so it distinguishes no matches from bad input fixes the whole class.

The organizational habit that follows is simple and rarely adopted: read a sample of real traces every week, including successful ones. Successful traces show wasted steps, unnecessary tool calls, and context bloat that never becomes a visible bug but quietly determines cost and latency.

Common failure modes

• Policy expressed only in prompts. The most common and most consequential mistake in the layer.
• No step or spend limit. A loop that cannot terminate itself will run until an external constraint stops it.
• Tool output treated as instruction. Retrieved content is data. Anything else is an injection path.
• Unbounded tool output. A tool returning a large file fills the context, evicts the actual task, and produces incoherent behavior.
• State only in process memory. A restart loses long-running work, and there is no way to resume or inspect it.
• Skills and tools defined in harness-specific config. Portable in theory, rewritten in practice.
• Silent failure. The agent reports success because the final model call said so, while a tool three steps back returned an error nobody surfaced.
• No approval path. Every action is either fully automatic or fully manual, with nothing in between, so teams choose manual and the system goes unused.

How to evaluate this layer

• Replaceable. If you moved to a different harness, how much would you rewrite? Skills, tools, and policy should mostly move.
• Inspectable. Can an engineer see the actual prompt, the tool calls, and the decisions, without adding instrumentation first?
• Portable. Are agent definitions, skills, and tool connections expressed in formats another runtime could read?
• Bounded. Are limits and permissions enforced in code, outside the model, and testable?
• Grounded. Does the harness supply facts from the data layer rather than relying on model recall?
• Auditable. Can you reconstruct a completed task from records alone, months later, including what was read and what changed?

A build sequence that works

1. Start read-only. Give the agent useful tools that cannot change anything. Most of the value and almost none of the risk.
2. Put limits in before the second tool. Steps, time, spend, and repeated-failure detection.
3. Record from the first day. The record is what lets you debug everything that follows.
4. Add write actions one at a time, each with a scope. Not a general write capability.
5. Add approval for anything hard to reverse. Show the proposed action, not a summary of it.
6. Move skills and tool definitions into portable files. Before you have three copies of them.
7. Introduce a broker only when there is a second executor. Not before.
8. Review the records regularly. Reading what agents actually did is the most reliable source of improvements.

What this layer is not

A harness is not a model with extra steps. The model contributes judgment. The harness contributes structure, limits, memory, and accountability, none of which the model can provide about itself.

A harness is also not the right home for durable business data. Task state belongs here. Facts belong in the data layer, where they can be queried, governed, and shared by systems that are not agents.

Finally, a harness is not a substitute for standards. Every harness will define skills, tools, and agent identity somehow. If those definitions live only inside it, the system is portable in principle and captured in practice, which is precisely the outcome an open architecture is meant to avoid.