Checking with GitHub before marking an AI agent's task done

On August 19, 2026, five cards on Omnigent’s board sat in done with their pull requests still open on GitHub, one of them four days stale. About 2,380 lines of work stranded on open PRs behind cards the board called done. Two months later, on the first full day of Jarvis v2, Omnigent’s successor, the fleet’s scorecard put throughput at seven to eight merges an hour. v2 merged 19 in the 14:00 UTC hour and 20 in the 15:00.

The done column and the scorecard had the same fault. Each was written by something that hadn’t seen the outcome. The rule I run on now: status is written by the system that observed the outcome, never by the agent that did the work.

With one agent in a terminal, the person at the keyboard is the observer. They see the diff land and the PR merge, and that is a real check. On October 11 my fleet started 47 lanes and merged 315 PRs. Nobody watches that. The observer has to be a step in the pipeline.

Five done cards, open PRs

In July, an Omnigent card moved when an agent called the board’s move tool with a column name. The board checked that a merge had been recorded. It never checked the merge. The check written on August 19 says the record-merge tool “validates only the shape of the pr_url … and then takes the caller’s word for the outcome.” The caller was the agent that did the work.

A manual gh pr list sweep found the five cards. The same check names the real gap: “nothing on a cadence had ever asked the question.” Two changes went in. Record-merge now asks GitHub before a card can ship, and refuses when GitHub doesn’t answer. Its own error text is the rule: “A review-lens’s or coordinator’s belief that a PR merged is not enough on its own: record-merge requires GitHub’s own state to read MERGED before a card can ship on it.” And the question became a sweep that runs on a schedule instead of whenever someone remembers to ask.

The same bug, both directions

It goes both ways. An agent “says that it’s completed work and it hasn’t really completed work.” Or it “completes writing the code, but it doesn’t send a marker at the end saying that it’s done, and so then it hangs.” Every row below is a report that was stored as the state of the work.

A false done strands work. A false fail relaunches it on unchanged inputs and pays for the tokens again. A missing marker leaves a lane hanging until something restarts it. How a reviewer earns a pass is its own note. This one is about who gets to write the pass down.

I also swap models in and out of stages, so whatever habit one model has about reporting, the next one may not share. “Those kinds of things should probably be more deterministic and done from a different perspective than just letting the agent do it.”

Events aren’t enough

Jarvis v1, from October 1 to 10, had one script per edge. By edge I mean the deterministic step between two agent stages: the claim, the review, the gate, the merge. verdict.sh printed pass or fail for a PR’s current head, and its own comment called it the “Source of truth for review state.” But the lane agent still closed its own card. finish.sh <id> merged <pr> <merge_sha> appended the agent’s line to the event log, and nothing asked GitHub. The scorecard counted those lines.

v1 merged its last PR at 11:57 UTC on October 10, and v2 merged its first at 15:03. The scorecard kept reading v1’s log. That was two faults in one number. It read the wrong stream, and its formula, v1 merges divided by hours since v1’s first claim, was a running average that could never read zero. Fig. 01 is what that looked like the next day: a nearly flat line drifting from 7.5 to 6.8 while v2 merged 315, a mean of 13.1 an hour.

I didn’t catch it, and no alarm did. A Codex review of the wave-2 plan did: “The scorecard parses v1 events. … Calling it more frequently cannot establish the requested v2 throughput or cost.”

The same day, agent summaries misread the rate in both directions. The plan put v2 at “~2 [merges an hour] since 16:22.” Seven merges between 16:22 and 17:50 is 4.77 an hour. Earlier, the 33 at 02:00 looked like a rate until I asked, and the agent answered: “The 33 was never a real rate. It was a one-time backlog flush.” Deriving status from events is necessary. It isn’t enough. The events have to come from the live pipeline, and every number names its window.

What writes status now

In v2 the merge script refuses unless GitHub says the PR is open at the head that was reviewed. After the merge it asks GitHub for the merge commit and writes that into the event log:

m=$(gh pr view "$PR" -R $R --json mergeCommit -q .mergeCommit.oid)
echo "... v2merge pr=$PR head=${SHA:0:10} merge=${m:0:10} gated_base=${GB:0:10} why=$WHY caller=$MCALLER" >> "$EV"

The merge commit in that line is GitHub’s. The agent that asked for the merge doesn’t supply it.

Verdicts became a closed set on October 17: PASS, PASS-pending-gate, FAIL, NONE, GATE-FAIL, GATE-CONFLICT, WAIT-PARENT, MAIN-RED, matched by an anchored pattern instead of a substring search. Run against the 3,997 review lines already in the log, the old match and the new one disagreed on none. The scorecard has one routine writer, and it computes from v2’s events.

An agent’s words still come in. They come in as typed inputs that trigger a check, not as state. A builder’s “skipped” is a hold, not a failed try. An error status from an agent “always triggers a read of the edge’s own record before any state change.” And every fallback to a default decision leaves an event someone can count.

Instruments get the same rule

The card ledger turns the event log into a history for each card. It last indexed at 19:21 UTC on October 13. From 19:26 every run failed on a timestamp it couldn’t parse, 3,344 runs in a row through October 26, with zero events indexed. Its alarm checked whether the process was alive. The process was alive. Its output was dead. The alarm fired once, on October 26, when the process itself died.

When I was asked about it, my answer was “Oh, I didn’t know, and I’m about to assign an agent to it.” It’s the September busy driver again, in the harness’s own observer: health judged by a claim of life, not by the newest thing written. The fix is the same rule. Alarm on the age of the last successful index, not the pid. As I write this, an agent is fixing it.

I’ve argued that autonomy expands with instrumentation. The ledger is the footnote. An instrument reporting on itself is one more agent saying it’s done. I don’t let the agent that did the work write its status, and the things that watch the work don’t get to write their own health either.

Underneath

The Omnigent quotes are its own commit messages, error text, and invariant docstrings from August 19 to September 18, 2026. The v1 quotes are its archived queue scripts. The v2 merge script and the verdict set are the pipeline repository’s as of late October, and the design-rule quotes are the harness’s own design notes. The per-hour merge counts in Fig. 01 are v2merge lines in the fleet’s event log for October 11, and the scorecard line is its pre-fix formula recomputed against the v1 log at each hour’s end. The ledger figures are the indexer’s own log, October 11 to 26. The Codex plan review is dated October 11, 2026.

The pattern has older names. Martin Fowler’s Event Sourcing captures every change to application state as an event, so the state can be rebuilt from the log. The Kubernetes API conventions split spec from status: status fields “should be the most recent observations of actual state,” and “controllers are granted read-only access to spec but full write access to status.” That is the line I’m drawing — the agent writes intent, the step that observed the outcome writes status. One word clashes. The same Kubernetes document calls its controllers “level-based rather than edge-based.” My edges are level-based too, because verdict.sh re-reads the current head every time it runs. Here “edge” names a place in the pipeline, not a trigger style. Both sources accessed October 29, 2026.

More field notes

Start