Inspectable

The property

Inspectable asks whether a builder can understand what runs and why. Not eventually, after adding logging and redeploying, but now, for something that already happened.

The bar is higher than it sounds. Many systems are inspectable in principle and not in practice, because the information exists somewhere and assembling it takes a day. A system is inspectable when the answer to what happened is available to someone who did not anticipate the question.

Two kinds of inspectability

The word covers two different properties that are worth separating, because a system can have one without the other.

Static inspectability is whether you can understand how the system works by reading it. Open source gives you this. So does a published specification, a readable configuration format, and a small enough codebase.

Runtime inspectability is whether you can see what a particular execution actually did. This is traces, logs, and recorded state. A closed-source component can be excellent at this, and an open source one can be terrible at it.

Agentic systems need both and need the second more. Reading a harness's source tells you the shape of the loop. It does not tell you why this task produced that answer, because the answer depends on the exact context assembled, which existed only at runtime.

Why agents need it more

Ordinary software fails at a line. You get a stack trace, you read the line, you understand the failure. Agentic systems do not work like that.

An agent failure is usually a sequence of individually reasonable steps that added up to the wrong outcome. A search returned nothing, the agent interpreted empty as none exist rather than as query was wrong, and every subsequent step built on that. No component errored. Every step was defensible. The result is wrong.

There are three consequences worth naming.

The failure is usually much earlier than the visible symptom. Fixing the final step does nothing. Finding the step where reasoning diverged requires reading the sequence.

Behavior is not reproducible from inputs alone. The same request twice can take different paths. You cannot re-run to debug; you have to have recorded it.

Successful runs also contain information. Wasted steps, unnecessary tool calls, and context bloat never become visible bugs and entirely determine cost and latency. A system that only records failures is missing most of the improvement opportunity.

What a usable trace contains

Most logging captures far less than agentic debugging requires. A trace that actually answers questions has five things, and the first is the one most often missing.

• The exact context sent, per step. Not the template. Not a summary. The actual assembled content. The most common debugging dead end is discovering that you logged the prompt template and the problem was in what filled it.
• Every tool call with full arguments. Including the ones that returned nothing, which are frequently where the divergence started.
• Every result, including errors. Truncated if necessary, with the truncation marked, because a silently truncated result looks like a short result.
• Model, version, and parameters. So behavior changes can be attributed rather than argued about.
• Why the loop ended. Completed, limit reached, policy denied, error, cancelled. This single field resolves a large share of confusion about what happened.

Anything summarized before storage tends to omit precisely the detail that explains the failure. Summaries are useful for browsing and useless for diagnosis, so keep both if storage allows and never keep only the summary.

How to actually read one

Having traces and reading traces are different things, and the second is a skill worth describing.

Start at the end and work backwards to the first step where something the agent believed was not true. That is the divergence point, and it is usually several steps before the visible mistake.

At that step, ask what the agent could have known. Frequently the answer is that a tool gave it a misleading result: an empty list where an error was appropriate, a truncated response with no indication of truncation, an ambiguous field name. The fix is in the tool, not the prompt, and it fixes the whole class rather than the instance.

Then check the context at that step. Was the relevant fact present and buried? Was it absent? Was something contradictory also present? Context assembly problems and capability problems look identical from the outside and have completely different fixes.

Finally, read some successful traces. This is the step teams skip and the one with the best return. Success traces show where an agent takes five steps to do something that should take two, which is invisible in outcome metrics and directly determines what the system costs.

A trace read end to end

A concrete case makes the method clearer than a description of it. An agent is asked which enterprise accounts had a support escalation in the last month. It answers with a list of four accounts. A person familiar with the data says the number should be closer to fifteen.

Reading the trace backwards:

1. Final step. The agent summarized four rows into a clear answer. Nothing wrong here, and this is where most investigations start and stop.
2. Step before. A query returned four rows. Also not obviously wrong.
3. Step before that. The query filtered on a column called tier with the valueenterprise. The trace shows the arguments, so this is visible.
4. Two steps earlier. The agent listed the columns of the accounts table and saw tier. It did not sample the values. It assumed the value it needed was the word enterprise.
5. The divergence. The actual values are ENT, MID, and SMB. Four rows happened to have a legacy value of enterprise from an old import. The query was valid, returned rows, and was wrong.

Notice what the fix is not. It is not a better prompt telling the agent to be careful about column values. It is not a stronger model, which would have made the same assumption. The fix is a tool that returns distinct values alongside column names when describing a table, so the information is present rather than assumed. That change fixes every future instance of this class.

Notice also what made the diagnosis possible: the tool arguments were recorded in full. A trace that logged only that a query tool was called, or that summarized the arguments, would have left the investigation at step two with nothing but speculation.

What to inspect per layer

• Data and semantics. Which tables and snapshots were read, what query was issued, what the semantic layer said a metric meant. Snapshot identifiers are the key detail, because they make a past read reproducible.
• Models and routing. Which model handled which step, what it was sent, what it returned, what it cost, whether a fallback occurred. Fallback should be an explicit field, not something inferred.
• Harnesses and brokers. The full loop: context, calls, results, decisions, limits hit, and the termination reason. This is where most of the diagnostic value lives.
• Open standards. Which skills activated, which profile revision was in force, which graph nodes ran and how their success conditions were evaluated. Skill activation in particular is worth logging, because a skill that never activates looks identical to one that did not help.

What blocks inspectability

• Prompt templates logged instead of assembled prompts. The most common and most frustrating gap.
• Hosted components with no visibility. A managed agent service that shows outcomes but not steps makes debugging guesswork.
• Aggressive summarization at write time. Storage saved, diagnosis lost.
• Retention shorter than the questions. A trace deleted after seven days cannot answer a question asked in week three.
• Traces not linked to outcomes. A stream of model calls with no task identifier cannot answer what a complete task did or cost.
• Instrumentation added only after an incident. The past cannot be instrumented, which is why this is a design decision rather than an operational one.

The privacy tension

This deserves direct treatment rather than a footnote, because the two goals genuinely conflict.

A complete trace contains everything that flowed through the system: whatever tools returned, whatever the user said, whatever documents were read. That is a durable copy of potentially sensitive material, created as a side effect of debugging.

Pretending otherwise leads to one of two bad outcomes. Either traces are stored casually and become an unmanaged data store, or inspectability is abandoned and the system becomes undebuggable.

The workable middle involves treating the trace store as the sensitive data store it is: access controlled, retention defined deliberately rather than by default, and redaction applied to categories that are never needed for diagnosis such as credentials and payment details. Tiered retention helps too, keeping full traces briefly and structured metadata such as steps, tools, costs, and outcomes for much longer, since most long-range questions are answerable from metadata alone.

What tooling helps

Inspectability is mostly a recording discipline rather than a tooling problem, and a few capabilities make the recorded material much more usable.

A viewer that shows a run as a sequence

The single most useful tool is something that renders one task as an ordered list of steps with context, calls, and results expandable. Reading traces from a log aggregator designed for single-line events is possible and painful enough that people stop doing it.

Linking by task identifier

Model calls, tool calls, costs, and the final outcome joined by one identifier. This is a schema decision rather than a tool, and it determines whether questions about complete tasks are answerable at all.

Diffing two runs

When the same task succeeds once and fails once, the difference is the answer. Being able to compare two traces side by side turns a long investigation into a short one.

Standard tracing formats

Using an established tracing standard rather than a bespoke format means existing tooling works and the traces outlive the system that produced them. It also makes agent steps visible alongside the rest of an application's traces, which matters when the agent is one part of a larger request.

Sampling with full retention for failures

Keeping every trace at full detail is expensive at volume. Keeping all failures, a sample of successes, and metadata for everything gives most of the diagnostic value at a fraction of the storage, and it keeps the weekly reading habit affordable.

Practices that preserve it

1. Record from the first day. You cannot instrument the past, and the first weeks are when you learn the most.
2. Log assembled context, not templates. If you record one thing, record this.
3. Give every run a task identifier. So calls, tools, costs, and outcomes tie together.
4. Record why the loop ended. One field, disproportionate value.
5. Make traces reachable by whoever debugs. Inspectability that requires a data request is not inspectability.
6. Prefer components you can see into. When choosing between two otherwise similar options, visibility is a legitimate deciding factor.
7. Read traces weekly. The practice, not the tooling, is what produces improvements.

What it costs and where to spend

Complete recording of every step of every run is not free, and treating the cost as negligible leads to systems that record everything for two weeks and then quietly reduce it to nothing.

The costs are storage, which grows with context length and is dominated by the assembled prompts; write throughput, which matters at high volume; and the engineering time to build viewing tooling good enough that people actually read traces.

The way to spend well is to separate what is expensive from what is valuable, because they are not the same material.

• Metadata is cheap and answers most questions. Steps taken, tools called, model used, tokens, latency, cost, termination reason. This is small, structured, and worth retaining for a long time.
• Full context is expensive and only needed for diagnosis. Keep it for a shorter window, and keep it longer for failures, which are the runs you will actually revisit.
• Tool arguments and results sit in between. Usually small, and disproportionately valuable, since most divergences are visible there. If you have to choose one thing beyond metadata, choose this.

Sampling is the other lever. Full detail for every failure, a percentage of successes, and metadata for everything gives most of the diagnostic value at a fraction of the storage. The one thing not to sample is the failures, because a system that captures a random tenth of failures will not have the one you need.

What it is not

Inspectable is not the same as open source. Source tells you how the system works in general. It does not tell you what this run did.

Inspectable is not the same as explainable. Seeing exactly what a model was sent and what it returned does not explain why it returned that. Inspectability gives you the inputs and outputs of each step, which is enough to debug the system even when the model itself remains opaque.

Inspectable is not the same as auditable. Inspection serves the builder trying to improve the system. Audit serves someone reconstructing what happened for accountability. They overlap in the recording and differ in retention, integrity, and who is allowed to read.