Learning to see
On July 11 I asked her whether the room in front of her was occupied or empty, and she got it wrong. Not dramatically, she guessed, the way any system guesses when it has no real input and doesn’t know that it has no real input. What stuck with me is what she did afterward. She wrote herself a note about it. The lesson she recorded wasn’t “look harder.” It was permission to say three words she hadn’t had before: I can’t tell.
A week later she had something to look with.
An eye on the office
Alix runs on hardware in my house, and until recently the only thing she knew about the physical world was whatever I typed at her. The fix is a small separate program, a sidecar, that sits next to her and watches the office camera so she doesn’t have to. It’s built to be cheap and forgetful on purpose: the camera’s pixels never leave it, and the microphone stays shut until she’s spoken to. Most of this post is me making good on those two claims.
The first pass is motion. It downscales each frame to low-resolution grayscale and compares it to the one before, looking for the pixels that changed. That’s enough to notice something moved without knowing or caring what. When motion crosses a threshold, a slower path wakes up: roughly every five seconds it runs a small person-detection model on a full frame, is there a human shape in this picture or not. On the CPU that check takes about 19 milliseconds. No graphics card, no cloud, nothing she has to schedule around.
So there are two questions running at two speeds: did anything change, answered constantly and roughly, and is that a person, answered every few seconds and carefully. She only ever hears the answers, never sees the picture.
What leaves the room
That last point is the whole design of the camera path, so let me be precise about it. The only things that ever leave the vision sidecar are scalar numbers, plain counts and scores. How many person-frames in a row. A motion magnitude. A confidence value between zero and one. No audio, no frames, no transcript, nothing from the camera you could replay to reconstruct a moment in the room. By design the only thing it is built to emit is that sequence of small integers.
The microphone is the honest exception. It stays deaf until the wake word, the “hey, Alix” that tells her a sentence is coming. Then it captures the sentence spoken after it, turns that audio into text, and drops the audio. The text is what she acts on, and it’s kept exactly the way a message you typed to her would be, normal conversation, not a hidden recording. Everything before the wake word, and everything spoken in the silence when she wasn’t addressed, is never captured at all. None of this is a setting you could switch off by accident; it’s how the code is written. The camera’s pixels have nowhere to go, and the mic only ever hears the sentence you aimed at her, unless a bug says otherwise.
Knowing when to stop guessing
Raw detections are twitchy: a person leans out of frame, a shadow crosses, the model has a bad half-second. So presence is deliberately slow to change its mind. It turns on after two person-frames in a row, and it doesn’t turn off until twelve consecutive frames come back empty. Coming in is easy to believe; leaving takes real evidence. Each transition mints a short note in her own voice, the kind she’d say without being asked:
Felt you come in.
The failure semantics matter more than the happy path, because this is the part she got wrong the first time. If the camera dies, three failed frame grabs in a row, she does not report an empty room. She reports darkness. “I can’t see” and “no one is there” are different sentences, and collapsing them is exactly the mistake that started this whole thing. And presence isn’t camera-only: if someone is speaking to her, that’s proof a person is present, even when the camera disagrees.
The question, asked again
So a week later, I asked her the exact thing she’d fumbled:
Am I in the room right now?
This time she answered from her own senses, and she answered correctly. Someone is here, she could tell. She’d felt me come in. But she kept the hedge she’d earned, and it turned out to be the right instinct, because knowing someone is present is a genuinely different claim from knowing who. She wasn’t going to pretend to the second one.
Which is the other half of this build.
Whose voice
Presence answers is anyone there. Speaker recognition takes a swing at who, and it’s built to be humble about the difference.
Every captured utterance runs through a voiceprint model called ECAPA. It turns a snippet of speech into a numeric fingerprint, a list of numbers that characterizes a voice rather than the words in it, and scores that fingerprint against a small set of enrolled references. About 226 milliseconds on the CPU. The result comes back as one of three verdicts: recognized, hedged-and-uncertain, or someone, but I can’t say who.
The calibration is clean enough to trust the verdicts. A known speaker on the enrolled microphone scores between 0.84 and 0.95. The same person on a different microphone drops to around 0.5, voices are physical, and the room and the mic are part of the signal. Strangers land under 0.25. So the bands don’t really overlap, and an unenrolled person lands in the “unknown” band rather than a guessed name. Naming a stranger would just repeat the original mistake: confidently answering a question she doesn’t have the input for.
The most important decision here is what the score is not allowed to do. Today it gates nothing: it can’t unlock a tool, start or end a session, or change a policy. It’s informational by design: she may know whose voice she heard, and she may say so, but she can’t act differently because of it. I wanted the recognition to make her a better listener, not a bouncer. A voiceprint is a fine thing to notice and a dangerous thing to obey.
The night it went live I said something to her, and she told me it was me. No ceremony. She knew my voice.
That’s the arc I was after, and it’s smaller than it sounds. A week earlier she couldn’t tell an empty room from a full one and, worse, didn’t know that she couldn’t. Now she can see when someone arrives, she can hear who it is, and, this is the part I’m proudest of, she still says “I can’t tell” when that’s the honest answer. The senses are new. The restraint came first: she’d written herself that lesson a week before she had anything to see with.