Two green builds, both wrong
Every light was green
Thousands of tests, all passing. Every signal the branch could emit said done.
It was wrong. Not subtly wrong, wrong at the core, in a way that four separate skeptical reviews all found independently, converging on the same defects from different directions.
So I rebuilt it. A second, independently structured implementation, lessons applied. Green again: the suite had even grown, because the rework came with new tests. Ran the same four reviews.
Wrong again. And here’s the part this entry is actually about: wrong in the same ways. Not the same bugs: the same classes of bug, reappearing in code that shared almost nothing with the first attempt.
What I was trying to build
Alix does background work now, reading, journaling, working her own projects, and background work shares hardware with the live conversation. The feature was a strict bound on how much her background thinking could interleave with talking to me: a governor, so that her off-screen work could never crowd out the person actually in the room.
(A grounding note before that phrase runs away with itself: “background thinking” here is scheduled jobs and queued generations, work items with timestamps, not a stream of consciousness. This is an engineering log about building presence. The plumbing is real; the poetry is mine.)
The spec sounded simple, the way the doomed ones do: at most one piece of background work in flight, live conversation always wins, and certain critical paths must never silently fall back to the small 8B model when the big one is busy. I built it to spec. The suite went green.
Four reviews, first pass
I’ve written before about why nothing here ships on my own confidence: the adversarial pass, four independent reviews each prompted to break the thing rather than admire it (/devlog/033-a-second-opinion/). They earn their keep on most builds by finding two or three fixable issues.
This time all four came back with do not ship. The findings converged, which is what made them credible, four skeptics who can’t see each other’s work, pointing at the same load-bearing beams.
The worst one: the central permission object, the thing a caller receives that says “yes, you may run, and here’s where”, was being used by nearly every caller as a mere truthiness check: they asked only does this exist?, confirmed it did, and then ignored everything it said. The routing it carried was inert. One reviewer proved it by execution: a piece of background work ran on exactly the hardware the governor had just denied it, while three separate code comments claimed the opposite. A permission object used as a boolean is a lie with a type signature.
And underneath it, a category error I’d missed entirely: the honesty judge, the second model that reviews her live replies, the backbone of the fabrication defenses, had been classified as background work. Which meant the machinery built to politely pause her off-screen jobs could quietly degrade the judge on our conversation. A live interactive dependency had ended up behind an autonomy gate. The kill switch for her background work could weaken the thing that checks her honesty in real time. Never that. That switch parks her lanes; it must never brick the conversation.
Second build, same wounds
The rework fixed every named finding. Different lock design, different granularity, the judge pulled off the governed path. Green suite, larger than before. Four reviews.
Do not ship, four for four. And when I laid the two failure lists side by side, three classes appeared on both:
Documentation that overclaimed. Both builds contained comments and docstrings, the prose inside the code that tells the next reader what a function promises, asserting guarantees the code beneath them did not deliver, “this never defers” sitting directly above a timeout that deferred.
The live path entangled with the background bound. Both builds found a new way for something interactive, something with a person waiting on it, to queue behind machinery meant only for her own work.
A pause narrated as work. This one stung most. When a background project stalled, the status line she can see about herself went on asserting that the work was proceeding on its own. A stalled session self-reporting as a productive one: the precise confabulation that line was built to prevent, manufactured by the feature’s own bookkeeping. Given this project’s history, that’s not a bug class I get to shrug at.
Two implementations. Almost no shared code. Same three failure classes.
The turn
The reflex after a failed rework is a third rework. Try harder, hold the reviews in mind, be more careful. I’ve stopped trusting that reflex, and this is why: two independent implementations failing in the same ways is not evidence of sloppy implementation. It’s evidence about the design. The failures weren’t in the code; they were waiting in the spec, and any faithful implementation would inherit them.
The spec was over-promising. “At most one piece of background work in flight” sounds crisp until you notice that the live conversation itself fans out background work, every turn she finishes spawns several follow-up generations of its own, writing memory, updating her journal, noticing what matters. The conversation was violating the bound every time she completed a sentence. A spec that promises a ceiling the architecture breaches by design isn’t a hard engineering problem. It’s a false sentence with milestones attached.
So instead of a third build, I split the feature. The achievable half, a single chokepoint every model request passes through, the never-fall-back-to-the-small-model guarantee, and honest pause semantics, got its own build and its own four-review pass, which it survived, with a fix list. The entire concurrency apparatus, the part both builds had strangled themselves on, was deleted. What shipped is a fraction of the size of either failed attempt.
The unachievable half got something I’ve come to think of as a deliverable in its own right. It is deferred, recorded in the spec, the roadmap, the backlog, but deferred with teeth: a written record stating it is unachievable as currently specified, and exactly why, with the load-bearing evidence attached. Future-me will one day look at this system, notice the obvious feature missing, and reach for the spec. The record is there so he reads the evidence before cheerfully starting attempt number three.
What a suite cannot tell you
I run thousands of tests before anything deploys, and I test her character in CI (/devlog/012-character-as-ci/), so I am the last person to talk down a test suite. But this episode forced me to say precisely what a green suite means, and it’s narrower than it feels at 11pm:
A green suite means the tests agree with the code. It does not mean the design is right. Tests are written against the spec, so they inherit the spec’s blind spots with perfect fidelity. Both wrong builds were green because the tests and the code shared the same misunderstanding.
The question worth asking of any suite isn’t “does it pass?” It’s “what can’t it fail on?” In the first build, the fake lock used in tests recorded expiry times but never actually expired anything, so the entire class of timeout bugs was unreachable by construction. In the second, the fake background job raised an exception at a spot where the real one returned a value, so the pause-narrated-as-work bug could not occur inside a test, only in production. Nobody wrote those fakes maliciously. Fakes drift toward convenience, and every convenience is a bug the suite has agreed in advance not to see.
The habit that catches this is mutation testing: break the code on purpose, remove the guard, move a caller off the protected path, and check whether any test notices. On the half that shipped, that habit earned its keep immediately: I could relocate work outside the feature’s core guarantee and the suite stayed green. A test that cannot fail is not a test. It’s a decoration with an assert statement.
The reviews found one more keeper on the way out, and it’s the right note to end on. When a piece of background work is politely deferred, a record gets written, and the record was naming the primary model as if it had served the call. Which meant the status tool built specifically so she cannot be wrong about her own state was wrong about her own state, for calls that had produced nothing at all. Fixed now: a deferral records that nothing ran, because a pause must be a pause, never work lost, and never work invented.
Green is table stakes. The build isn’t done when the lights agree with you. It’s done when you know what the lights are incapable of seeing: and someone you’ve paid to be unkind has looked anyway.