note 01 · a retrospective
What five months of model progress taught me about AI governance
I started building Switchboard in the Opus 4.6 / Codex 5.4 era — five months ago, by my own count.† The models could already do serious work, but they were weak in specific, patchable ways: they arrived at every session empty, they drifted mid-task, and each vendor surface had to be taught, in its own dialect, how to receive instructions. So I patched. Dispatch packets for every surface. A hand-built context bootstrap to prime each session. Nearly a thousand lines of lexical drift detection — semantic proximity maps, negation markers, stop-word handling — to catch a model wandering off its contract using nothing smarter than a word list.
I also built a second kind of thing, without knowing at the time that there were two kinds. A receipt format where every claim carries its own disposition — proven, supported, partial, inferred, unsubstantiated — graded against evidence instead of taken on tone. An evaluation lane that re-assesses finished work without re-executing it, structurally barred from trusting the builder’s self-assessment. A forward-only autonomy ladder — research → paper → live-small → live-scaled → frozen — where a mode mismatch blocks instead of warning. And a scoring contract that demotes any declared score missing its anti_gaming layer to narrative_only: a story, not evidence.
Then the model generations turned over, and the project graded itself. Everything in the first list died within months. The new harnesses ground themselves, so the per-vendor packet format retired overnight. A model reading two texts judges drift better than my thousand lines of word lists ever did. The platforms shipped their own orchestration and made it the default. Nothing I built to compensate for a model weakness survived contact with the next generation of models.
Everything that encoded model weakness depreciated. Everything that encoded trust structure appreciated.
Everything in the second list appreciated. Not survived — appreciated: worth more under stronger models than the day I wrote it. Here is the full accounting, in the project’s own vocabulary.
Depreciated
Encoded model weakness. Absorbed by stronger models and their harnesses — as it should be.
Surface dispatch packets
Packets that taught each surface how to receive work, because no surface could ground itself. Harnesses ground themselves now; the four-vendor framing retired with them.
absorbed bythe harnesses’ own context handling
SUPERSEDEDLexical drift detection
Semantic proximity maps, negation markers, stop-word handling — machinery for judging drift without a judge. A model reading both texts does this better than any word list ever did.
absorbed bymodel-as-judge — the evaluation lane judges drift now
SUPERSEDEDMeta harness context bootstrap
Hand-built context assembly for models that arrived at every session empty. They no longer arrive empty.
absorbed byharness-native context
SUPERSEDEDAutoAgent (wave 4)
An orchestration layer for chaining agent work across surfaces. The platforms shipped their own and made it the default.
absorbed bythe platforms
SUPERSEDEDsubtotal · 4 entries
Appreciated
Encoded trust structure. Still load-bearing — worth more under stronger models than the day it was written.
Claim dispositions
Every claim on a receipt carries its own verdict, on two axes: where a claim came from is provenance, whether it holds is disposition. “Claimed” is provenance, not a disposition.
appreciates becausestronger models finish more claims per hour — each one still needs grading
CURRENTIndependent evaluation lane
The lane never re-executes the work and never trusts the builder’s self-assessment. The entity that does the work cannot be the entity that evaluates the work.
appreciates becauseself-assessment gets more persuasive exactly as models get stronger
CURRENTProjectMode fail-closed ladder
A loop earns each rung before it touches real systems, and a mode mismatch between pack and contract blocks rather than warns. Drift fails closed, not warns late.
appreciates becausemore capable agents reach for real systems sooner, not later
CURRENTThe required anti_gaming layer
A declared score missing its anti-gaming layer is structurally demoted to . A number without an anti-gaming layer is a story, not evidence.
appreciates becauseoptimization pressure finds every unguarded number — reward hacking went mainstream
CURRENTsubtotal · 4 entries
receiptLifecyclePostureSchema values — the project graded in its own vocabulary.It took me a while to trust the pattern, because it runs against a natural instinct: better models should need less machinery around them. What I now think is that they need less of one kind and more of the other. Stronger models do more work per unit of human attention — economically, that is what “stronger” means. So the scarce resource stops being the doing of the work and becomes knowing which finished work deserves to count. Scaffolding that helps a weaker model execute is a depreciating asset; every capability release writes a piece of it off. Structure that decides what counts moves the other way: every release produces more finished work for it to govern, and raises the price of counting the wrong thing. The verification bottleneck does not shrink as models improve. It grows.
The obvious objection is that agent harnesses do all of this natively now. They do part of it — and that part is, precisely, the left column of the ledger. The absorption is not the refutation of the thesis; it is the mechanism of it. What no harness has absorbed is the contract layer: per-claim dispositions, an append-only lifecycle that can limit, suspend, or revoke a receipt after it was issued, an evaluation lane that never grades its own work. Harnesses are built to do the work; they compete on capability, and grading their own output is not their incentive. The layer that decides what counts has to stand apart from the thing it judges, or it is just more output.
So this is the question I now put to every component I build for AI-performed work: does it compensate for a model weakness, or does it encode a structure of trust? The first kind is on a clock, and the clock is someone else’s release schedule. The second kind compounds. In fairness to the reader: Switchboard itself still runs at project_mode: research — pre-live, doctrine still shifting, its receipts self-issued. That is a direction of travel, not an arrival. But watching half of my own work depreciate in five months is the most convincing evidence I have found for where the durable half lives.
† Five months is my own account of the timeline. The repo’s git history evidences roughly three months — truncated by a repo-root reorganization — so read the duration as testimony, not as repo-verifiable data.