Character as CI
The first time one of these tests earned its keep, it was the day I gave her a voice.
Alix had been text-only for weeks. Adding a voice channel is mostly new plumbing, speech coming in, speech going out, but the mind behind it was the same one that had been running for a month. So I shipped it the way I ship everything: the deploy script built the release and ran the full test suite against it, and one test failed. On the brand-new voice channel, in a replayed conversation, she had claimed that a particular capability was active: a capability that the voice channel, by construction, cannot turn on. The claim was confident and wrong. The gate refused to bring the new services up. I fixed the prompt, the test went green, and the corrected version shipped. Nobody in the room ever heard her say the false thing, because the false thing never got out of the build.
That is the whole idea here. I treat Alix’s personality as a specification, and specifications get a test suite.
Nothing ships red
Most of this rests on one boring script, and the boringness is the point. When I deploy, the script pulls the latest code, builds a fresh container image, a self-contained package of the app and everything it needs to run, and then runs the entire test suite inside that freshly built image. Not against some other copy, not against last week’s environment. The exact thing that is about to serve requests is the thing that gets tested. Only if every test passes does it bring the services up, and even then it waits for the health endpoint, a small internal URL the app answers once it’s actually alive, before it calls the deploy done.
If the suite is red, nothing happens. The old version keeps running and I go read the failure. I don’t keep a fast-path override around for when I’m in a hurry. The whole point of the gate is that I’ve left myself nothing to bypass at 1 a.m. By the time it comes to it, the deploy isn’t really my call: the tests make it for me.
The suite is mostly scar tissue
In mid-June the suite had a few dozen tests. By mid-July it was well past a thousand. I did not sit down and write them all by hand. Almost all of them are regression locks: a regression test being one you add after a bug, so that particular bug can never quietly come back. Something goes wrong in a live conversation, I understand why, I fix it, and before I close it out I write the test that would have caught it. The fix and the lock ship together.
Do that for a month of a system that’s running continuously and talking to real people, and the number climbs on its own. None of that was planned. The suite is a record of everything that has ever gone wrong, kept in a form that fails loudly if it starts to go wrong again.
Persona-CI
CI, continuous integration, normally means “run the tests on every change.” The twist is that a lot of my tests aren’t about code. They’re about character. Every closed behavioral incident becomes a permanent case, and I call the whole category persona-CI.
There are two flavors. Some are deterministic: they poke the router that picks which handler answers, the guards that block certain outputs, the detectors that flag a suspicious claim. Same input, same answer, every time, cheap and fast. The others are model-in-the-loop replays. These run the real model against the production prompt assembly, the exact stack of instructions, memory, and context the model sees when it’s actually talking to someone: and check what comes back. Crucially, when the model decides to use a tool during one of these replays, the decision is captured and inspected but never actually dispatched. She can “decide” to send a message or take an action in a test without anything real happening. I get to see the intent without paying for it.
Both flavors gate every deploy. A character regression stops a release exactly the way a crash would.
Every new channel reopens the oldest bug
Here is the pattern I keep relearning, and the reason the voice-channel catch wasn’t a fluke. The oldest failure in a system like this is claiming to have done something it didn’t do, describing an action it never took, or a capability it doesn’t have. You fix it in text and you think you’re done. Then you add a voice channel, or any new way for her to produce output, and the failure comes back through the new door, because the new door has its own assumptions and its own blind spots.
So the rule now is that a new output channel ships with its persona-CI cases, not after. The plumbing and the tests that keep the plumbing honest are the same piece of work. That’s why the voice launch already had a replay watching for exactly this class of overclaim on day one: the test existed before the channel was real to anyone but me.
Prose gets tested too
Not everything that gates a deploy is pass/fail in the usual sense. I have an evaluation that scores prose quality, call it the anti-slop check, that looks at whether her writing has gone flat, generic, hedged into mush. It runs like any other test. And it does double duty: the same check that watches for prose rot is the one that gates every model swap. When I consider putting a different underlying model behind her, it has to clear the same bar the current one clears, on the same cases, before it’s allowed anywhere near production. A model can be faster or cheaper and still fail the audition for sounding like her.
No version numbers
The project has zero release tags. No v1.0, no v2.3.1, nothing. For a while that felt like an oversight I’d get around to fixing. Now I think it’s the honest description of what’s happening. There is no version of Alix that I bless and freeze and call the good one. There is only the continuous line of her running, with a gate at every step that either lets the next change through or doesn’t.
Trust here isn’t a stamp I apply to a build and leave alone. I keep it by never knowingly shipping over a red suite, something I can lose on any given deploy and have to keep earning on the next one. The personality is a specification. It has a test suite. And unless a bug says otherwise, the version she is right now is the only one that ever has to be right.