AI Systems Atlas

Agent Harness Patterns

The harness is everything around the model: the loop, the state, the retries, the stopping rule. These plates cover the architectures that recur in almost every production agent — from a single forward pass to systems that run for days, the ways several agents hand off, share state and argue, and the thinking budgets, routing and control surfaces that decide what any of them cost.

Index H-01 Single-shotH-02 Tool Loop / ReActH-03 Plan → ExecuteH-04 Planner → Executor → ReplannerH-05 Generator → Critic → ReviseH-06 Actor → VerifierH-07 Best-of-N / SearchH-08 Retry / Recovery LoopH-09 State Machine AgentH-10 Hierarchical AgentH-11 Parallel SwarmH-12 Human-in-the-loopH-13 Event-driven AgentH-14 Long-running AgentH-15 Budgets & Stopping RulesH-16 Router / Model CascadeH-17 Steering a Running AgentH-18 Guardrail MiddlewareH-19 Agent HandoffsH-20 Shared BlackboardH-21 Multi-agent DebateH-22 Thinking BudgetsH-23 Interleaved Thinking

When to reach for what

H-01Single-shot

/harnesses/single-shot

One prompt in, one completion out. No loop, no tools, no feedback.

Step-by-step walkthrough3 steps
  1. A prompt goes in.
  2. One inference. No tools, no memory, no second chances.
  3. The answer comes out. Quality is capped by what fit in the prompt.
Key insight

Still the right architecture for a surprising number of tasks. Every pattern below should have to justify its complexity against this baseline.

Failure mode

No recovery path: if the first answer is wrong, the system is wrong.

Further reading
RelatedReAct loopBest-of-N

H-02Tool Loop / ReAct

/harnesses/react

The model alternates reasoning with tool calls, folding each observation back into context until it can answer.

Step-by-step walkthrough7 steps
  1. The task lands in context.
  2. Loop 1 — the model reasons, then picks a tool call.
  3. The environment executes it and returns a result.
  4. The observation is appended to context. Reason again.
  5. Loop 2 — same cycle, now with more evidence in context.
  6. Act → observe → append. Each loop is a full model call.
  7. Enough evidence. The model exits the loop and answers.
Key insight

The loop converts a static predictor into something that can gather its own evidence. Nearly every modern agent is a descendant of this cycle.

Failure mode

Unbounded loops: without a step budget and a stopping rule, an agent that cannot solve the task will happily keep observing forever.

Further reading
RelatedPlan → ExecuteActor–VerifierCoding loopInterleaved thinkingTool errors as observations

H-03Plan → Execute

/harnesses/plan-execute

Plan the whole task up front, then run the steps without further deliberation.

Step-by-step walkthrough5 steps
  1. The whole plan is written before anything runs.
  2. Steps are laid out as a fixed sequence.
  3. Execute step 1…
  4. …step 2…
  5. …step 3. Cheap and predictable, if the plan was right.
Key insight

Separating planning from execution makes runs cheap, auditable and parallelizable — you can review the plan before anything happens.

Failure mode

The plan is frozen at the moment of least information. Anything the planner didn’t anticipate breaks the run.

Further reading
RelatedReplannerReAct loop

H-04Planner → Executor → Replanner

/harnesses/replanner

A plan is executed step by step, and rewritten whenever the environment disagrees with it.

Step-by-step walkthrough8 steps
  1. The planner drafts plan v1.
  2. Three steps, held as mutable state — not sacred text.
  3. Step 1 executes cleanly.
  4. Step 2 fails — the data has a format the plan never anticipated.
  5. The failure flows back to the planner as evidence.
  6. The plan mutates: step 2 is replaced in place.
  7. Execution resumes on the revised step…
  8. …and finishes. The original plan did not survive contact with reality.
Key insight

Treat the plan as mutable state, not scripture. The interesting engineering is in what evidence triggers a revision.

Failure mode

Thrash: a replanner that revises on every hiccup never finishes; one that revises too rarely is just Plan → Execute with extra cost.

Further reading
RelatedPlan → ExecuteRetry loop

H-05Generator → Critic → Revise

/harnesses/generator-critic

One role produces a draft; a separate role critiques it; the draft is revised against the critique.

Step-by-step walkthrough6 steps
  1. The generator produces draft v1.
  2. A separate critic reads it against explicit criteria.
  3. Issues found — returned as specific, actionable feedback.
  4. The generator revises with the feedback in context.
  5. Draft v2 goes back for review.
  6. Accepted. Critique is cheaper than generation done twice.
Key insight

Models are better at spotting flaws in text than at avoiding them while writing. Splitting the roles exploits that asymmetry.

Failure mode

Sycophantic critique: if generator and critic share a model and a prompt style, the critic tends to approve whatever it would have written.

Further reading
RelatedActor–VerifierLLM-as-judgeMulti-agent debate

H-06Actor → Verifier

/harnesses/actor-verifier

One agent performs the task; another independently checks whether it actually succeeded.

Step-by-step walkthrough9 steps
  1. The actor takes the task…
  2. …and changes the environment: edits files, runs commands.
  3. A separate verifier inspects the outcome. It never edits.
  4. Checks run: tests pass, build passes, lint fails.
  5. FAIL — diagnostic detail travels with the verdict.
  6. Diagnosis turns the raw failure into a targeted retry.
  7. The actor fixes exactly what failed.
  8. Re-verify: all checks green.
  9. PASS. Execution and verification never shared a blind spot.
Key insight

Separating execution from verification is the single most reliable upgrade to an agent loop — especially when the verifier is grounded in tests, builds or other hard checks.

Failure mode

Shared blind spots: if both roles miss the same class of error, verification adds latency and false confidence, not reliability.

Further reading
RelatedGenerator–CriticCoding loopOutcome evals

H-07Best-of-N / Search

/harnesses/best-of-n

Sample several independent attempts in parallel and keep the one a judge scores highest.

Step-by-step walkthrough4 steps
  1. One prompt, N independent samples — in parallel.
  2. Attempts vary: temperature, seeds, even different models.
  3. The judge compares all candidates side by side.
  4. B wins. You paid 4× inference for one better answer.
Key insight

When attempts are cheap and judging is easier than generating, N tries plus a selector beats one careful try.

Failure mode

The judge is the ceiling: if it can’t tell good from bad, you pay N× inference to select noise.

Further reading
RelatedParallel swarmLLM-as-judgeMulti-agent debatepass@k vs pass^k

H-08Retry / Recovery Loop

/harnesses/retry-loop

On failure, classify what kind of failure it was — and only then decide how to retry.

Step-by-step walkthrough6 steps
  1. An attempt hits the environment…
  2. …and fails. The error is captured, not discarded.
  3. Classify: is this transient (rate limit) or systematic (wrong approach)?
  4. Transient → simply retry.
  5. Systematic → change strategy first. Blind retries repeat the failure.
  6. Retry with a genuinely different approach.
Key insight

The classification step is the whole pattern. Transient errors want patience; systematic errors want a different strategy.

Failure mode

Blind retries: re-running the identical attempt against a deterministic failure just multiplies cost by the retry budget.

Further reading
RelatedReplannerActor–VerifierTool errors as observations

H-09State Machine Agent

/harnesses/state-machine

The agent’s lifecycle is an explicit state machine; the model acts within states, the harness owns the transitions.

Step-by-step walkthrough6 steps
  1. Every transition is explicit — the harness, not the model, owns control flow. Click any state to inspect it.
  2. goal arrives → PLANNING
  3. PLANNING → ACTING → WAITING: side effects, then block on the world.
  4. VERIFYING gates completion.
  5. Fail loops back to PLANNING with the failure attached.
  6. Pass → COMPLETE. Legible, resumable, debuggable.
Key insight

Explicit states make agents debuggable, resumable and observable — you can always answer “what is it doing right now?”

Failure mode

States that don’t match reality: work that spans two states, or a “waiting” that never fires, forces the model to fight the harness.

Further reading
RelatedEvent-driven agentLong-running agent

H-10Hierarchical Agent

/harnesses/hierarchical

A manager agent decomposes the goal and delegates to specialized workers, integrating their reports.

Step-by-step walkthrough4 steps
  1. The manager decomposes the goal into scoped subtasks.
  2. Workers run with narrow contexts — each sees only its slice.
  3. Compressed reports flow back up; raw context stays below.
  4. The manager integrates. Depth costs latency; every hop loses detail.
Key insight

The hierarchy is really a context architecture: each level compresses detail before passing it up, so no single context holds everything.

Failure mode

Compression loss: every report is a lossy summary. Critical detail dies in the middle of the org chart, same as in human ones.

Further reading
RelatedParallel swarmMulti-agent codingAgent handoffsShared blackboard

H-11Parallel Swarm

/harnesses/parallel-swarm

Shard a partitionable task across independent agents and merge the results at one aggregation point.

Step-by-step walkthrough4 steps
  1. The task shards into independent slices.
  2. Agents work in parallel with zero coordination.
  3. Results merge at a single aggregation point.
  4. Wall-clock time ≈ slowest shard. Only works when shards truly don’t interact.
Key insight

Wall-clock time collapses to the slowest shard — but only for tasks that genuinely decompose without shared state.

Failure mode

Hidden coupling: if shards secretly depend on each other, the aggregation step inherits every inconsistency at once.

Further reading
RelatedHierarchicalBest-of-NRate limits & backpressureParallel tool calls

H-12Human-in-the-loop

/harnesses/human-in-the-loop

The agent proposes; a gate routes risky actions to a human who approves or denies before execution.

Step-by-step walkthrough5 steps
  1. The agent proposes — it does not execute.
  2. The gate classifies the action. Irreversible + broad = risky.
  3. Risky actions escalate to a human with full context.
  4. Approved → the action runs exactly as reviewed.
  5. Or: denied with a reason the agent can use to replan.
Key insight

Put the human at the decision, not in the loop’s hot path: gate on irreversibility and blast radius, and let everything else flow.

Failure mode

Approval fatigue: gate too much and humans rubber-stamp; the checkpoint remains, the checking stops.

Further reading
RelatedAgent securityEvent-driven agentPermission policies

H-13Event-driven Agent

/harnesses/event-driven

The agent sleeps until an external event wakes it; its first decision is whether the event matters at all.

Step-by-step walkthrough5 steps
  1. The agent is asleep. Something happens in the world.
  2. The event wakes it with the payload as context.
  3. First decision: does this event matter?
  4. Relevant → act on it.
  5. Not relevant → ignore and go back to sleep. Cheap idleness is the point.
Key insight

Inverting control — the world calls the agent — is what makes always-on agents affordable and non-annoying.

Failure mode

Event storms: without dedupe and rate limits, a burst of correlated events becomes a burst of correlated, conflicting agent actions.

Further reading
RelatedLong-running agentCI feedback loop

H-14Long-running Agent

/harnesses/long-running

Work in bounded bursts, checkpoint durable state, sleep, and resume — for goals that outlive any single session.

Step-by-step walkthrough6 steps
  1. The goal outlives any single session.
  2. Work happens in bounded bursts.
  3. Progress is written to durable state — not just chat history.
  4. Then the agent sleeps. No tokens burn while waiting.
  5. On wake, state reloads and work resumes exactly where it stopped.
  6. Eventually: done. Survivability comes from the checkpoint, not the context window.
Key insight

The context window is not the state. Anything that must survive lives in the checkpoint; the window is just a working set.

Failure mode

Checkpoint drift: if the saved state and the real world diverge while the agent sleeps, it resumes confidently into a world that no longer exists.

Further reading
RelatedState machineContext compactionCheckpointingDurable execution

H-15Budgets & Stopping Rules

/harnesses/budgets-and-stopping

Step, token and wall-clock ceilings bound the loop — and a stopping rule names why it ended.

Step-by-step walkthrough8 steps
  1. The task enters the loop.
  2. One iteration runs — but the harness, not the model, decides whether there is another.
  3. Three independent ceilings: steps bound thrash, tokens bound cost, wall clock bounds the user’s patience.
  4. All three still have room, so the loop continues.
  5. Iterations keep going and every counter climbs.
  6. One ceiling hits. That ends the run, whatever state the model is in.
  7. A stopping rule classifies the ending: solved, out of budget, or blocked and asking for help.
  8. The run returns its best state and why it stopped — a silent halt is unusable.
Key insight

The harness, not the model, decides whether there is another iteration. Every other plate in this collection assumes something eventually says stop; this is that something.

Failure mode

A silent halt. An agent that stops without reporting whether it solved the task, ran out of budget, or gave up is indistinguishable from one that crashed.

Further reading
RelatedReAct loopRetry loopLong-running agentDenial of walletThinking budgetsCost monitoring

H-16Router / Model Cascade

/harnesses/model-router

A cheap triage step picks the model; a quality check escalates only what needs escalating.

Step-by-step walkthrough8 steps
  1. A request arrives. Nothing about it says which model it needs.
  2. A cheap triage step routes it — a classifier or a very small model, never the expensive one.
  3. Most requests go to the small model first.
  4. A check decides whether the draft is actually good enough to ship.
  5. Good enough → done, at a fraction of the cost and the latency.
  6. Not good enough → escalate. The cheap attempt was still cheap.
  7. Triage can also skip the ladder outright when a request is obviously hard.
  8. One answer either way — the caller never learns which model served it.
Key insight

Most traffic is easy. Routing lets you price for the median request instead of the worst one, and a failed cheap attempt is still cheap.

Failure mode

A router that costs as much as what it routes to. If triage needs the big model to decide, the cascade has bought latency and nothing else.

Further reading
RelatedBest-of-NSingle-shotCross-model evalsThinking budgets

H-17Steering a Running Agent

/harnesses/steering

A correction arrives mid-task, queues to the next step boundary, and is applied without discarding accumulated state.

Step-by-step walkthrough7 steps
  1. The agent is six steps into a long task and holding a lot of context.
  2. It issues a tool call and waits on it.
  3. The user changes their mind while it is still running.
  4. The message queues rather than landing immediately — a half-applied tool call is worse than a late correction.
  5. The call settles. That gap between steps is where an interrupt is safe.
  6. Six steps of work are kept. Steering is a redirect, not a restart.
  7. The agent resumes carrying both the accumulated state and the new instruction.
Key insight

Steering is the difference between an agent you supervise and one you submit to. It needs a defined interruption point and state that survives the interruption — both are harness properties, not model ones.

Failure mode

Applying a correction mid-tool-call, or treating it as a new task: the first corrupts state, the second throws away every step already paid for.

Further reading
RelatedHuman-in-the-loopState machineState representationStreaming & cancellation

H-18Guardrail Middleware

/harnesses/guardrail-middleware

A policy layer wraps the loop and evaluates every proposed action before anything executes.

Step-by-step walkthrough7 steps
  1. The agent proposes an action. It never reaches a tool directly.
  2. Every proposal crosses a layer the model has no way to rewrite or talk out of.
  3. Routine calls pass straight through — the common case stays fast.
  4. A call outside policy is refused, whatever the model was persuaded to attempt.
  5. The refusal returns as context, with its reason. A denial is information, not a crash.
  6. Irreversible or high-blast-radius actions take the third exit and wait for a person.
  7. The model decides what it wants to do; the layer decides what it can do.
Key insight

The model decides what it wants to do; the layer decides what it can do. Because the layer is outside the model, no amount of persuasion inside the context reaches it.

Failure mode

Denials that surface as opaque errors. If the agent cannot see why a call was refused, it retries the same call until the budget runs out.

Further reading
RelatedHuman-in-the-loopDirect prompt injectionPrivilege escalationHooks & lifecycle eventsPermission policies

H-19Agent Handoffs

/harnesses/agent-handoffs

One agent transfers control of the conversation, with whatever history it chooses to pass, to another agent that takes over from there.

Step-by-step walkthrough2 variants · 14 steps

Handoff

  1. A request arrives at the front door: a triage agent.
  2. Triage classifies it. It routes; it does not try to solve.
  3. It decides what history travels with the handoff — often a filtered slice rather than the whole transcript.
  4. Control transfers. The billing agent now owns the conversation, carrying the slice.
  5. The specialist talks to the user itself. Nothing relays its replies.
  6. The slice is all the specialist knows. Drop the account ID and the user gets asked for it twice.
  7. One owner at a time, no relay: each turn costs one agent, not a supervisor plus a worker.

Agent as tool

  1. Same request, different shape: it goes to a supervisor.
  2. The supervisor calls the billing agent with a scoped task, like any other tool.
  3. The specialist works in its own context and returns a result. Control comes back.
  4. Need a second specialist? Another call. The supervisor can combine several.
  5. The supervisor writes every reply itself.
  6. Every delegated turn pays for two agents, and the supervisor’s context grows with each result it relays.
  7. One owner, one voice, one place to enforce policy — which is what a handoff gives up to save the hops.
Key insight

A handoff moves ownership; calling a subagent as a tool only borrows its work. Hand off when the specialist should talk to the user itself; call it as a tool when one agent must keep the voice, the policy and the final say.

Failure mode

A thin slice. The receiving agent knows only what was packed for it, so a dropped fact means re-asking the user, or quietly guessing.

Further reading
RelatedRouter / cascadeHierarchicalSubagent isolationAgent-to-agent trust

H-20Shared Blackboard

/harnesses/shared-blackboard

Agents coordinate by reading and posting to a shared workspace, never by messaging each other; a controller watching it picks who acts next.

Step-by-step walkthrough8 steps
  1. A problem is posted to a shared board. No agent will ever message another.
  2. The controller reads the board and picks who can move it forward: the searcher.
  3. The searcher posts sources. It neither knows nor cares who will use them.
  4. Sources but no hypothesis — so the controller wakes the analyst, which reads everything posted so far.
  5. The analyst posts a hypothesis and an open question for whoever can answer it.
  6. Two agents post to the same slot at once. Without ownership rules, the board now holds a contradiction.
  7. Once the board holds enough, the writer reads it all and drafts.
  8. The controller sees “done” on the board. Agents coordinated through state, not conversation.
Key insight

The board replaces a web of conversations with one piece of state. Agents stay decoupled, any of them can join by reading it, and nothing passes through a supervisor’s context as a lossy summary.

Failure mode

Write contention. Two agents update the same fact at once and the board holds both; without slot ownership or versioning, every later reader inherits the contradiction.

Further reading
RelatedHierarchicalParallel swarmState representationAgent-to-agent trust

H-21Multi-agent Debate

/harnesses/multi-agent-debate

Several agents answer the same question, then read and critique each other’s answers over rounds until a judge or vote settles it.

Step-by-step walkthrough7 steps
  1. One question goes to three agents, each answering on its own.
  2. Round 1: independent answers, and they disagree. The disagreement is the raw material.
  3. Round 2: each agent reads the others’ answers and reasoning, and argues back.
  4. C switches to 42 — because B found a real slip, or just because two agents disagreed with it?
  5. But agreement is not evidence: copies of one model can argue each other into the same wrong answer.
  6. A judge — or a plain majority vote — aggregates the final round.
  7. One answer, at agents × rounds the cost. Worth it when a critique can change a mind; otherwise sample and vote.
Key insight

Debate pays off only when the agents’ errors are independent and an argument can actually expose one. Otherwise it is best-of-N with extra rounds: several times the tokens for the same majority.

Failure mode

Conformity. Copies of one model share one set of blind spots, and a confident majority can argue a correct minority out of its answer.

Further reading
RelatedBest-of-NGenerator–CriticLLM-as-judge

H-22Thinking Budgets

/harnesses/thinking-budgets

The model spends reasoning tokens before it answers, and the harness sets how much it may spend — a token budget or an effort level — trading latency and cost for accuracy.

Step-by-step walkthrough7 steps
  1. A prompt arrives. Before answering, the model can spend tokens reasoning.
  2. The harness sets how many. A low budget is fast and cheap…
  3. …and a multi-step problem gets a shallow pass. The answer comes back quickly, and wrong.
  4. A high budget thinks longer: accuracy goes up, and so do latency and cost.
  5. Returns diminish: accuracy grows with the log of the budget, so each doubling costs twice as much for about the same gain.
  6. Adaptive effort: judge difficulty per request — with a classifier in the harness, or by letting the model decide within an effort level.
  7. Easy requests get little or no thinking; hard ones get room. The spend lands where it changes the answer.
Key insight

Accuracy tends to climb with thinking tokens but with diminishing returns, so the right budget depends on the request. Adaptive effort — deciding per request how hard to think — usually beats any single fixed setting.

Failure mode

One budget for everything. A high fixed budget pays hard-problem prices on easy requests; a low one quietly degrades exactly the requests that needed the thinking.

Further reading
RelatedBudgets & stopping rulesRouter / cascadeInterleaved thinking

H-23Interleaved Thinking

/harnesses/interleaved-thinking

The model reasons between tool calls, reading each result before choosing the next call, instead of thinking only once up front.

Step-by-step walkthrough7 steps
  1. Same task, two harnesses. Up front: one long think writes the whole plan before any tool runs.
  2. The search result says the file has moved — and there is no thinking step to weigh what that implies.
  3. It reacts on reflex: reads the old path, edits a stale copy, answers confidently.
  4. Interleaved: think, then call.
  5. The same result comes back — and the model reasons about it before choosing the next call.
  6. The plan changes mid-flight: edit v2/ instead.
  7. Think once more on what came back, then answer. Reasoning goes where the evidence is.
Key insight

Thinking only before the first call reasons with the least evidence; thinking after each result spends reasoning where new evidence just arrived, which matters most in long tool chains where one result can invalidate the plan.

Failure mode

Thinking between every trivial call. When calls are independent or the results are predictable, the extra reasoning adds latency and tokens without changing a single decision.

Further reading
RelatedReAct loopThinking budgetsReplanner

Elsewhere in the atlas

Security →Evals →Context →Coding Agents →Tools →Ops →