Companion to The agent is the easy part. The article explains what I learned building and running the harness. This is the technical version: how the loop works, what a campaign looks like, the decisions that turned out to matter, and the ways it broke.
This isn't a framework and there's nothing to install. It's one Python file, some YAML campaign files and an append-only log. PyYAML is its only Python dependency. Everything else it shells out to is already in use by the repo it works on: git, the GitHub CLI, dbt and Claude Code. There's no daemon and no database.
I started building it less than two weeks ago and have changed it continuously as it broke in ways I hadn't expected. The useful part isn't really the code. It's the decisions around it.
This describes the harness as it exists in August 2026. It isn't intended as a reference architecture for agent systems. It's the current shape of a small system built for one dbt repo, by one operator, after running it hard enough to find some of the places where the obvious version breaks. That's also why I've left the rough edges in.
The examples below are structurally representative and deliberately anonymised. The real campaign files contain internal model names and production table paths.
The basic unit of work is a campaign step: one small, predefined change that should result in one reviewable pull request.
A tick looks at one campaign and makes one decision about what should happen next:
tick:
read the campaign
rebuild its current state from the log
find the first step eligible to move
decide ONE of:
launch | wake | wait | terminate | escalate
record what is about to happen
do it
record what happened
exit
A tick does not sit around supervising an agent. It makes one decision and exits.
An automatic sweep runs ticks across campaigns in priority order. Campaigns can run in parallel, but each campaign has at most one agent working at a time. Nothing stops two campaigns from touching the same file.
Campaigns are YAML files. A campaign is one piece of related work, split into steps that run one at a time. Each step is one change with its own target and its own verification.
Steps are not necessarily the same change repeated. Sometimes they are: the largest campaigns apply one transformation across 13 to 20 places. But most are much smaller, and some are a dependent sequence: measure the problem first, then act on what the measurement found, then retire the old thing. Of 83 campaigns currently running, 33 have a single step and 25 have two.
Here's a synthetic example based on the real structure:
name: account_entity_repoint
priority: 6
target_repo: <path to dbt repo>
worktrees_root: <path for worktrees>
base_branch: production
branch_prefix: "harness/account-entity-repoint/"
blurb: |-
Re-point consumers of the source-shaped `accounts` table onto `account_entity`,
one consumer at a time, with row-level parity proven against production.
facts: |-
`account_entity` is a superset of `accounts` on account_id.
Three columns were renamed:
accounts.name -> account_entity.account_name
accounts.created -> account_entity.created_at
accounts.is_deleted -> account_entity.deleted_flag
Do not widen scope to the consumers under dbt/models/finance/.
Different owner, separate campaign.
steps:
- path: dbt/models/sales/mart/account_summary.sql
branch_no: 1
precondition:
pattern: "ref\\('accounts'\\)"
must: present
why: >-
If the old ref is no longer present, there is nothing for this step to do.
target: |-
COMMIT BEFORE YOU BUILD.
Swap `ref('accounts')` for `ref('account_entity')`.
Apply the column renames from facts.
Do not change filter logic.
Commit the change before building or comparing.
Push verification results in a second commit.
override: |-
This model feeds a dashboard that reads account_name positionally.
Verify that column specifically, not just row counts.
- paths:
- dbt/models/sales/mart/account_health.sql
- dbt/models/sales/mart/account_health.yml
id: account_health
branch_no: 2
precondition:
pattern: "ref\\('accounts'\\)"
must: present
target: |-
Swap the ref and rename the columns.
Keep the unique test on account_id intact.
override: |-
This consumer filters on is_deleted.
Verify parity with that filter applied after the rename to deleted_flag.There are four parts I care about.
facts contains everything I already know that I don't want every agent rediscovering. Mappings, measured facts, known exceptions, things explicitly out of scope. I write it like I'm briefing someone competent who has never seen the repo. When a fact turns out to be wrong, I correct it in the campaign rather than expecting future agents to rediscover the mistake.
A precondition answers: is this step still valid? For a ref migration, that can be as simple as checking that the old ref() still exists. That also makes the step naturally resistant to running twice. Once the migration is done, the thing that made the step eligible is gone.
The target says what the agent is actually supposed to change. These are deliberately narrow. The agent is not being asked to "clean up this model." It is being asked to make one known transformation and prove that it preserved the result.
This is where I initially tried too hard to abstract. Different consumers often need different verification. One needs row-level parity. Another needs a particular filter checked. Another has ordering semantics. Another needs a specific column compared.
I had one 17-step campaign where all 17 steps needed different verification instructions. That isn't a bad default. The variation is the content. Trying to abstract it away made the campaign shorter and less useful.
The log is just an append-only sequence of intents and outcomes. An intent is written before an external action:
{
"type": "intent",
"action": "launch",
"campaign": "account_entity_repoint",
"step": "account_summary"
}Then the action happens. If it succeeds:
{
"type": "outcome",
"action": "launched",
"campaign": "account_entity_repoint",
"step": "account_summary"
}Current state is derived from those records. The outcomes I currently care about are roughly:
| Outcome | Meaning |
|---|---|
launched |
an agent started a new step |
woken |
an agent returned to an existing PR |
merged |
the good terminal state |
punted |
the agent declined the step and explained why |
reopened |
something previously settled became active again |
interrupted |
a run died and was reconciled |
source_gate_refused |
a precondition rejected the step |
agent_failed |
the agent subprocess failed |
The log currently gets replayed from the beginning on every tick. That is fine at this scale. It obviously won't be fine forever. If it becomes a problem, I'd add periodic snapshots and replay only what happened after the latest one. I haven't built that because I don't need it yet.
Most of the functions that decide what happens next operate on ordinary dictionaries. They answer questions like:
- What state is this step in?
- Has CI passed?
- Are there comments that still need an answer?
- Has this verdict already been dealt with?
- Which step should move next?
They don't need git, GitHub or a worktree to answer those questions. That separation matters because I want only one part of the system to be unpredictable: the agent. The harness should be boring. Give the decision logic the same log state and the same snapshot of the pull request and it should make the same decision.
I violated this once by calling gh pr view halfway through the decision function. That made the logic unnecessarily painful to test because the function was both reading the outside world and deciding what that world meant. Moving that GitHub read to the beginning fixed most of it. Fetch the PR once, turn it into a dictionary, then let the rest of the decision operate on that. No framework needed.
These are representative of what I'm currently running:
max_concurrent_campaigns: 20
max_concurrent_agents: 20
max_launches_per_window: 40
launch_window_seconds: 3600
max_open_prs: 40
auto_update_branches: true
auto_ready_for_review: true
request_copilot_review: true
copilot_review_wait_seconds: 1800
ci_absent_wait_seconds: 1800
max_attempts: 10
agent_timeout_seconds: 5400
agent_model: sonnet
agent_effort: highThe individual values matter less than a few interactions between them.
I keep max_concurrent_campaigns and max_concurrent_agents equal. If campaigns are lower than agents, some agent capacity can never be used. If I allow more campaign ticks than agent slots, a tick can inspect a pull request, decide what should happen, and then wait for an agent slot. By the time the slot opens, the pull request it looked at may have changed. Keeping the limits equal means that if a tick is allowed to make a decision, it can act on that decision immediately.
max_open_prs limits how much review work can exist at once. But that alone allows a sweep to create the entire allowance in one burst. So there is also a flow limit: how many new PRs may be launched in a given window. I learned that distinction after filling the stock limit in one sweep.
The harness can give an agent 90 minutes and still have another part of the system kill it after 10. That happened to me during a warehouse comparison that normally takes longer than the runner's default background-wait ceiling. So the timeout on the harness is not enough. Any runner-level timeout needs to be derived from it or at least checked against it.
There's also one thing I don't time yet. A campaign can be ready to run but sit starved of capacity indefinitely, and nothing complains.
These are all things that actually happened.
I originally derived step identity from its position in the campaign. Then I changed a 15-step campaign into a 21-step campaign while one of its PRs was open. Step 2 no longer meant the same thing. That is enough to orphan or misinterpret work already in flight.
Now step identity is declared explicitly rather than inferred from list position. The broader rule is: changing a campaign definition must not change the meaning of work that already exists.
I still don't know why. The lock makes it harmless, so I stopped caring very much. The more useful lesson was that external actions should tolerate being attempted twice. That property is usually called idempotency. I would rather make retries harmless than build the entire system around the assumption that something executes exactly once.
At one point the test suite spawned child processes that re-imported the harness with the real configuration and real credentials. It opened a real draft PR. The tests had mocked the dangerous values inside the test process. The child processes didn't inherit those mocks.
Lesson: protect the boundary where a process gets its configuration and credentials, not only the code inside the test.
I maintained a separate status document by hand. Eventually it said a campaign had never run while the log showed five launches and five merges. The status document was another copy of state, and copies drift. The log already contains the truth. Generate status from it.
I record the resolution, but I didn't originally record who actually made the decision. Later I wanted to know what percentage of escalations were handled without me. I tried inferring it from the resolution text, note length, latency and time of day. None of those methods gave me a trustworthy answer.
The fix is one field:
resolver: agent | humanIf you care who made a decision, record it when the decision happens. Don't try to reconstruct it later.
There is an ambiguity after a process disappears. Suppose the log contains:
intent: launch step 7
but no outcome. Did the process crash? Or is it still working?
A file lock doesn't answer that after a crash because the operating system releases the lock when the process dies. I solve it with a small lease written into the intent:
intent["lease"] = {
"pid": os.getpid(),
"host": socket.gethostname(),
"started_at": now,
}Then reconciliation can ask:
def is_live(intent, grace=300):
lease = intent.get("lease") or {}
if lease.get("host") != socket.gethostname():
return False
if age(lease["started_at"]) > AGENT_TIMEOUT_SECONDS + grace:
return False
try:
os.kill(lease["pid"], 0) # signal 0 is an existence check, nothing is sent
except ProcessLookupError:
return False # no such process
except PermissionError:
return True # it exists, I just can't signal it
return TrueThere's no heartbeat thread. On one machine, the process ID tells me whether the worker still exists, and the age limit protects against treating a reused process ID as the original run forever.
I err on the side of treating an ambiguous process as live. Waiting unnecessarily is bounded by the age check; incorrectly declaring a live process dead could interfere with work that's still running. If this ever becomes a multi-machine system, I'd need something stronger.
One design choice has made later fixes much cheaper: external side effects already go through a small number of functions. Agent processes start in _spawn_agent. Log records go through _append. Campaign validation goes through load_campaigns. That means adding a lease or making an external action safe to retry usually means changing one chokepoint rather than finding twenty call sites.
The rule I use now is:
Before adding a new layer, check whether the behaviour belongs at an existing boundary.
For example, I still want one wrapper around GitHub mutations so that comment posts, thread resolutions and other external actions can all get the same retry protection.