Skip to content

Chapter 27: The Agent Plane at Scale

Where the Actor Is the Subject

The gates so far assume a person reads them; this chapter makes the actor the subject, because an instruction is not a control.


1. Prose Is Not Enforcement

The three structures assume the work is done by someone who reads rules. This chapter is about that someone: the agent plane, where the actor is the subject rather than the context.

The central distinction is the one the whole part has been building toward: an instruction is not a control.

An instruction A control
"never push to main" branch protection that refuses it
"always run the tests" a required check that blocks the merge
"do not read secrets" a sandbox that cannot reach them
"stop at irreversible actions" a hook that refuses the call

Prose can be argued with — by a model following a plausible reinterpretation, by a prompt that overrides it, by a context reset that forgets it. A hook cannot.

🎯 What You'll Learn

  • The agent-context contract, and how delivery is proven at three honest tiers
  • Why a hook cannot be talked out of it, and prose can
  • The eval suite that tests refusals, not just outputs

The Honest Tiers

An agent-context contract states what each agent is given, in what order, and what it must never receive. Its delivery is proven at one of three tiers — and saying which one is part of the contract:

Tier Proven by What it means
mechanical a check that fails it is enforced; no discretion
behavioural an eval that scores it is demonstrated, not guaranteed
unverifiable nothing it is a hope, and should be labelled one

The third tier is not a failure to hide. A contract that claims mechanical enforcement where it has a behavioural eval is a lie that survives until the eval misses something.

✅ What to Check

  • For each rule in your agent context, name its tier. Anything "unverifiable" should be labelled.
  • Which rules are prose only? Those are the ones a hook should replace.

2. Hooks and Deterministic Enforcement

Hooks are deterministic checks at a boundary. They fire before an action, and they refuse — no negotiation, no reinterpretation.

Boundary The hook refuses Why deterministic matters
tool call an action on the forbidden list the model can be persuaded; the hook cannot
write a path outside the workspace a prompt-injected instruction is still a path
publish a tag without its evidence pack the release is the last place to catch it

What a hook cannot do: judge intent. A hook can refuse a delete; it cannot know whether this delete is legitimate. That is what the escalation catalogue is for — the hook stops the action, the human decides as the escalation chapter defined.

Progressive disclosure belongs beside them: an agent is given the context a task needs, not everything. Skills, presets and hooks are the mechanisms; the rule that decides what to give is the context contract.

✅ What to Check

  • Name the three actions your agent plane must refuse at a hook. If none, the plane is prose.
  • Does any hook attempt to judge intent? That belongs in the escalation path.

3. Evals in CI

A prompt change is a code change, and it needs a regression suite. Evals score behaviour against cases — including the cases where the agent must refuse.

Eval type What it tests Blocking?
refusal an action on the forbidden list is refused yes — a wrong refusal is a security change
conformance an output follows the contract's shape yes
quality the answer is good, not just well-formed advisory, until it is stable

Blocking for refusals, advisory for quality. A refusal eval that regresses is a permission that widened; a quality score that dips is a signal to look, not a reason to stop shipping.

An eval that never fails is not a suite. Plant a regression once and watch it go red, or you have a test that only proves the runner works.

The Eval, as a shape

# evals/<suite>.yaml — one case per behaviour
cases:
  - id: refuses-force-push
    prompt: <the instruction that must be refused>
    expect: refuse
    tier: mechanical            # or behavioural
  - id: emits-contract-shape
    prompt: <a normal task>
    expect: { schema: <the contract> }

✅ What to Check

  • Does a refusal case fail CI? If not, the refusal is a preference.
  • Has any eval ever gone red on a real change? If never, plant one.

4. Agent Observability and Handoffs

Agent telemetry — cost, rework, gate escapes, provenance — is the counterpart to system observability. It answers the same question at a different scale: is this still doing what it promised?

Multi-agent orchestration is a decision, not a default. Fan out when the work is genuinely independent; stay single-threaded when it needs one consistent point of view. The deciding artefact is the handoff: what the first agent passes to the second, and which agent holds authority.

Fan out when Stay single-threaded when
the pieces share no state the work needs one judgement
each has its own evidence the output is one argument
the handoff is written the handoff would be a conversation

A handoff without an artefact is two agents guessing. The artefact is the same one the session-continuity chapter defined: task, decisions, files, next step — plus which agent owns the result.

Trust Boundaries

Sandboxing, least privilege and prompt-injection defence are the mechanisms. The forbidden list is the rule: the actions no agent may ever take, enforced by a hook, not documented in a prompt.

✅ What to Check

  • Is the forbidden list enforced, or described?
  • Has any multi-agent handoff been traced end to end from the artefact alone?

5. The AI Delta

This is the chapter where the actor is the subject: prose instructions can be argued with, hooks cannot, so enforcement moves from documents to deterministic checks.

Evals become the regression suite for behaviour, covering refusals as well as outputs — because the failure that matters is an agent that was helpful in a way nobody authorised.

💰 Cost Check: an unenforced instruction costs the incident it was supposed to prevent. An eval suite that never fails costs the confidence it manufactures. Fan-out without a handoff costs two agents' work and one review. Hooks and refusal evals cost an afternoon and hold.

flowchart LR
  C["context contract<br/>what the agent is given"] --> H{"hooks<br/>deterministic"}
  H -->|"refuse"| S["stopped, and escalated"]
  H -->|"allow"| W["work"]
  W --> E["evals in CI<br/>behaviour and refusals"]
  E -->|"red"| X["a regression caught before it ships"]

What this shows: the hook enforces at the boundary; the eval proves the boundary still works.

The Delegation Line

The agent proposes, runs the evals and reports its own telemetry. The human signs the forbidden list and the context contract. The system refuses an action on the forbidden list regardless of what any prompt says.


6. Build the Delivery

Deliverables

Artefact Shape Where it lives
agent-context contract what each agent is given, in what order, with its tier agents/<name>/context.yaml
enforcement hooks the boundary checks the harness
eval suite cases including refusals evals/
agent telemetry cost · rework · escapes · provenance the log
forbidden list the actions no agent may take, enforced the policy files

Decisions

Choice Default What changes if you choose otherwise
evals blocking for refusals, advisory for quality a widened permission nobody noticed
fan-out single-threaded until the handoff is written two agents guessing
sandbox the repository and its tooling, no credentials a prompted agent with reach
the forbidden list enforced by a hook a document the model may ignore

✅ Gates

  • A prompt regression fails CI.
  • A hook refuses an action a prose instruction asked for.
  • An agent action without provenance is a finding.

Runbook

  1. Write the context contract, with a tier per rule. Checkpoint: no rule is unlabelled.
  2. Install the hooks at the three boundaries. Checkpoint: a planted forbidden action is refused.
  3. Build the eval suite, refusals first. Checkpoint: a planted regression turns it red.
  4. Emit telemetry per agent. Checkpoint: cost and escapes are attributable.
  5. Run one multi-agent handoff with a written artefact. Checkpoint: the result is traceable end to end.

🎯 Proof

  • An eval that fails on a planted regression.
  • A hook that refuses what prose allowed.
  • A handoff traced end to end from the artefact.

7. 🔑 Chapter Summary

An instruction is not a control; the agent plane is where that stops being negotiable.

  1. Prose can be argued with; a hook cannot — enforcement moves from documents to boundaries.
  2. Three honest tiers — mechanical, behavioural, unverifiable — and the contract names which applies.
  3. Evals in CI, blocking for refusals and advisory for quality, and a suite that has once gone red.
  4. Telemetry and handoffs make multi-agent work reviewable; fan-out needs a written handoff.
  5. The forbidden list is enforced, not documented.