Kompot: a robot that keeps eye contact, listens, and answers

I build robots in the evenings. Kompot is one of my most recent works: a wide face with two big eyes, a head that turns, an antenna that wiggles, a camera, a microphone array, and a little thermal printer. It notices when somebody walks up, and it talks to them. My children are eight and seven, and the eight-year-old asks it what a tyrannosaurus was. It tells him, puts a picture on its face, and offers to print it out.

That last sentence took three years and a complete rewrite. This is the story of both versions, including the parts that did not work.

What I actually wanted

Not an obvious assistant. There are plenty of those, and they all live in a cylinder that does not look at you. I wanted something with a face that reacts, that turns its head when you move, and that a child would want to talk to. Closer to a pet than to a speaker. Think Disney Imagineering creations.

Which sets the bar in an awkward place. A smart speaker is allowed to take two seconds and answer in a flat voice. A thing with eyes is not: if it does not react while it is thinking, it reads as broken rather than as busy. Most of the engineering below is about reducing response latency or hiding latency with some animatronics tricks.

What it does today, and what it still cannot do

What already works:

• It boots asleep, wakes when called by name, and stays in the conversation afterwards, so the second question does not need the wake word.
• It watches the room and follows whoever is in it with its eyes, and turns its head to whoever is speaking.
• It answers out loud and in writing, in the language of the person it is talking to, and its mouth moves while it speaks.
• It knows who is talking, by voice.
• It looks things up on Wikipedia and on the web, puts the answer and a photograph on its face, and prints them on the thermal printer.
• It reads the house and changes it: temperature, presence, calendar, music, and the lights, blinds and windows.
• It draws pictures, takes photographs, and describes what it can see.
• It plays six games on its own face with a PS5 pad, and picks one from a gallery of covers.
• There is a web panel for all of it: every conversation, every voice, every setting, the logs, and a power button.

What is still pending:

• It does not move. The chassis has wheels and the motors are sitting in it unwired, so this is a robot that turns its head. Everything below is queued behind that, because navigation needs wheel odometry and there is no odometry without turning wheels.
• Its wake word is a stock one, hey_marvin. Training its own reproducibly is not solved.
• It tracks left and right but not up and down yet.
• The chassis has to be redesigned for better airflow and to add wheels.
• It will have a SCARA-like arm, in the sense that the first joint exists as a printed part and the rest still needs to be designed and fitted.

The first version: one Python app and a face made of pictures

I wrote about 2,500 lines of Python across a handful of processes that talked to each other over MQTT. There was a broker on the robot and topics called touch, llm, screen and head. One process listened, one drew the face, one moved the servos.

It genuinely worked: it woke on a Polish wake word thanks to Picovoice, transcribed with Whisper, answered with gpt-3.5-turbo, spoke back with tts-1, looked at things with the camera, remembered conversations by summarising them into a JSON file at shutdown, and could print a picture on the thermal printer.

Two details from it I am still fond of. The wake words were Polish and there were three of them: ziemniaczek (little potato), kompot, and góra, all trained through Picovoice's Porcupine with the Polish acoustic model. And I did not draw the face. It was a set of pre-rendered PNGs, several of them generated with DALL·E, blitted with pygame and swapped when the mood changed.

The sprite face is the thing that dated fastest. Every expression you want is a file. Every blend between two expressions is a file. Wanting the eyes to look somewhere means a file per direction, and at that point you stop adding expressions.

The rewrite: onto ROS 2

What runs now is a Raspberry Pi 5, ROS 2 Jazzy, seventeen systemd services that come up at boot, two more when somebody picks up a gamepad, and two on the server in the attic.

Moving from MQTT to ROS 2 was not about the message bus. MQTT is fine. It was about everything that comes with ROS: typed messages instead of hand-parsed JSON, quality-of-service settings that let a video topic drop frames while a command topic does not, a launch system, parameters I can change on a running robot, and a tooling story where I can subscribe to any topic from my laptop and watch what the robot is actually doing.

That last one changed how I work more than anything else in this project.

$ ros2 topic list | grep kompot
/kompot/audio/level
/kompot/audio/say
/kompot/audio/speaking
/kompot/face/article
/kompot/face/caption
/kompot/face/expression
/kompot/face/gaze
/kompot/face/image
/kompot/face/viseme
/kompot/head/position
/kompot/photo/print
/kompot/voice/transcript

Two things in that picture are worth saying out loud. The robot is not self-contained and was never meant to be: transcription runs on a server in the attic with a CUDA graphics card in it, and the answer comes from OpenAI. But it degrades rather than dies. The attic server being unreachable costs it its ears, not its face, and that is a deliberate requirement rather than a happy accident.

And the house is reached through the same server, not directly. The robot is one more client of it, holding a token, and it is never trusted to say who it is speaking for. That last part matters more than it sounds, and it comes back later.

The face

The panel is a 7.9 inch strip, 1280×400, mounted sideways so the framebuffer is actually 400×1280 and every frame gets rotated on its way out. It is RGB565 straight to /dev/fb0, no compositor, no window manager, nothing between the renderer and the glass.

Nothing is a sprite any more. I draw the face with signed distance fields in numpy: an eye is a rounded rectangle function, the mouth is an arc, and each is evaluated over a coordinate grid into a single luminance plane. Colour goes on at the very end, through a lookup table, when the frame is packed to RGB565.

That last decision is the one that makes it affordable. One plane of float32 instead of three of uint8, and the entire tint of the face is a different LUT rather than a different render. The robot going from calm cyan to alarmed red costs nothing.

Because the pose is parametric, an expression is just a set of numbers, and blending between two expressions is interpolation. Eyes look somewhere by moving a pupil offset. There are no files.

A full redraw costs about 34 ms on the Pi, 24.8 ms to compose and 9.7 ms to pack. That is far too slow for 30 fps, and it does not matter, because the renderer hashes the pose it is about to draw and skips the frame entirely if nothing has changed. A robot sitting still with a caption on its face draws almost nothing.

The flip side is that anything which changes the pose every frame costs the whole budget. I wrote a breathing animation early on, fine enough to look smooth, and it cost 75% of a CPU core on its own, doing nothing but making the eyes rise and fall by two pixels.

Seeing

There is a Hailo-8L accelerator on the Pi's PCIe bus, fed by a Camera Module 3, running person detection at 26.9 frames a second. The detections drive the gaze: whoever is in frame gets looked at, and the eyes follow them.

I did not put the accelerator in for speed so much as for heat and headroom. The same detection on the CPU leaves nothing for the face, and the face is the most visible element of the robot.

Horizontal tracking works well. Vertical does not, and the bug is a good one: there are two independent causes, and fixing the obvious one changes nothing observable. I mounted the camera upside down and my code negates the x coordinate but not the y, clearly wrong, clearly a bug. But the vertical target is also the centre of a full-body bounding box, and a standing person's centre sits near the middle of the frame at any distance, so the value never leaves the dead band.

Hearing and speaking

The microphone is a reSpeaker XMOS XVF3800 array. It wakes on hey_marvin through openWakeWord, records until you stop talking, and ships the audio to a machine in the attic that runs Whisper. The Pi 5 could do it, but not while doing everything else while maintaining power and thermal under control.

To be exact about the attic, since it is doing more than it looks: large-v3, through faster-whisper on a graphics card, which transcribes four seconds of speech in 0.16 of a second. It also takes a voiceprint of the same audio on the way past, 33 milliseconds of work that rides home on the same message, and that is what lets the robot know which of us is talking.

Speech out is OpenAI's gpt-4o-mini-tts, streamed as raw PCM into aplay. Streamed, not buffered, because waiting for a whole sentence to render before playing any of it is a second of silence the child interprets as the robot having ignored them. First audio comes 0.78 seconds after the text is ready.

The single best measurement in this project is about echo. The speaker is on the microphone array, a few centimetres away, which sounds like an obvious mistake: the robot should wake itself constantly. It does not. Played at full volume, the phrase "hey marvin" out of its own speaker arrives at the capture side at −54.9 dBFS, below the −46.9 dBFS ambient noise of a quiet room, and scores 0.0000 on the wake word detector.

The array's echo cancellation is simply that good. Which means I never have to mute the microphone while the robot talks, which means you can interrupt it mid-sentence, which is most of what makes talking to it feel like talking rather than like taking turns with a machine.

One detail in that diagram is the sort of thing you only find by reading a datasheet twice. The array will not record unless it is also playing. Both its USB endpoints are marked synchronous and the echo canceller wants both halves running, so with nothing going out, 92% of every recording comes back as exact zeros.

It does not present as a broken microphone, which is what makes it expensive. The level meter reads a perfectly plausible figure, because the impulses that do survive are full scale. It sounds like static. Every service reports healthy. The tell is that real audio essentially never contains an exact zero, so the thing to count is zeros, before gain, before channels, before anything. There is now a service whose entire job is to stream silence into that speaker forever.

Making the mouth move

The mouth moves while it speaks, and how it does that is a small lesson in being honest about what you can measure.

Real lip sync means recognising phonemes. I am not running a phoneme recogniser for a mouth that is one arc on a 400 pixel panel. What a PCM stream gives you for free is loudness, so the jaw opens with the audio envelope and the shape is whichever of the face's existing mouth positions has about that opening. It is not lip sync. It reads as speech anyway, because the timing is real.

The part that had to be right is timing, and it is not obvious. The audio is written to the sound card as fast as it arrives from the network, which is far faster than real time, and I hold a buffer back on purpose so a network stall does not become a hole in the middle of a word. So a mouth shape published when its bytes are written runs seconds ahead of the sound.

I therefore stamp every shape in audio time (the byte offset divided by 48,000) and a thread sleeps until that moment actually arrives. Which in turn means the analysis cannot live in the loop that writes the audio, because sleeping there would stall the stream it is analysing.

It publishes about 8 shapes a second and costs 0.8% of a core.

The brain, and giving it hands

A transcript goes to an OpenAI model with a persona and a set of tools, and what comes back is both the sentence to say and the things to do while saying it. Transcript to caption is 3.2 to 3.5 seconds and a turn costs about $0.0008, which is roughly a tenth of a cent per question my son asks.

The tools are the robot's body and its senses: set an expression, turn the head, wiggle the antenna, check the battery, look through the camera and describe what is there, take a photo, print it, draw a picture. navigate_to is deliberately absent rather than a stub, a tool that accepts a call and does nothing teaches the model it moved when it did not, and then it starts narrating journeys it never took.

That picture is the answer to the question I get asked most, which is why a robot with a graphics card in the attic still takes four seconds. It is not the network, which is ten milliseconds. It is not transcription, which is a sixth of a second. It is the model, and the model is four fifths of the wait.

Which is why so much of this robot is about covering that gap rather than closing it. It says something short while it thinks. Its eyes stay alive. It shows you what it heard, small and dim, before it answers, so a wake word that fired on the dishwasher is visible rather than mysterious. None of that makes it faster and all of it makes it feel faster, which for something with a face is the same thing.

Then there are the tools that reach outside the robot. It can search the web. It can ask the house about itself, our home runs on KNX with a server behind it, and the robot talks to that over MCP on the local network, so it can answer how warm the bedroom is, whether anybody is home, what is on the calendar, and whether music is playing upstairs. It can turn a light on, and it refuses to between nine at night and seven in the morning.

Most recently I taught it to look things up on Wikipedia, which turned out to be much more interesting than it sounds.

Why "look it up on Wikipedia" is harder than it sounds

The plan was small: try Wikipedia before a general web search, because most of what an eight year old asks is an encyclopaedia question, and Wikipedia answers it free, faster, and with a photograph. Then show it on the panel in three columns, picture on the left, summary across the other two, and offer to print it.

Wikipedia's search ranks pages by how well they match the words you searched for. That is not the same as being the article about the thing, and the gap is not subtle:

• Ask about the Moon and it ranks Apollo 11 first. The Moon comes fourth.
• Ask about the giant panda and Kung Fu Panda comes second.
• Ask in English against the Polish Wikipedia about the Eiffel Tower and you get an article on English grammar, because that is what the English words matched.

Every one of those articles mentions the subject constantly. They are simply not about it. My fix is to fetch five candidates and weigh them by how much of the title is what was asked about, ties going to the plainest title.

And a Polish one I did not expect. Ask for aksolotlu and you get the axolotl. Ask opowiedz mi o aksolotlu, tell me about the axolotl, and you get nothing at all. Not the wrong article: zero results. One imperative verb takes the result count to zero. Inflection, which was my first suspect, turns out to be handled perfectly well.

So I ask the question twice, once as written and once as just its keywords, and weigh both sets of results together. After that it got 21 of 22 test questions right, and the one it missed was my expectation being wrong rather than its answer.

Printing has its own trap, and it is only visible on paper. The picture wants dithering, a thermal head has no grey, so without diffusing the error a photograph prints as a black blob with a white hole in it. But running the same dithering over text at 8 dots per millimetre scatters stray dots around every letter and the text stops being readable. And the automatic exposure that makes photographs printable will happily take a page that is mostly white and darken it until the printer lays a wash of toner over the whole thing.

So: I level and dither the picture on its own, threshold the text, and never level the assembled page as a whole.

The week it learned to play

Everything above took about three years. The next part took a week, and I want to be honest about why: the hard parts were already paid for. Once a face can draw anything, a brain can call tools, and a message bus connects them, a new capability is usually a module and two lines somewhere. Several of these are one file each.

It knows who is talking

The voiceprint that rides home with every transcript is compared against the people it has met. A recognised voice gets their own conversation history, their own name, and a note about who they are, which is the part that matters: tata, or a friend of Zosia's from school. That note is what stops it explaining a giraffe to their father the way it explains one to a seven year old, and for a visitor it is the only thing the robot could never have known.

Two decisions in it I would make again. It compares against a person's best three samples rather than an average, because these children are eight and seven and their voices are moving. And a match needs a margin over the runner-up, not just a good score, because the question is never "is this Zosia", it is "is this Zosia rather than her brother". When it is unsure it says so and files the conversation under nobody, which is much better than filing it under the likeliest child and quietly corrupting the one thing per-person history was for.

And answers each person in their own language

Every profile carries a language. Mine is the household default, so anybody it does not recognise gets answered in Polish; a guest set to German is answered in German, in speech, in captions and on the printed card. The robot has no language of its own. It has its owner's.

This one took four attempts, and every one of them taught me the same lesson from a different direction: you cannot fix a prompt by making it louder. A guest set to German, asking in Polish, was answered in Polish, from a prompt that said "answer in German" at the top and did not contradict itself anywhere. Bold did nothing. Moving it to the very top did nothing. What worked was making the instruction the last thing the model reads on every round, sent after the whole conversation rather than before it. Appended once, it stopped being last the moment a tool result landed behind it, which is exactly why it worked in testing and failed the moment somebody asked about the weather.

The other half of the same lesson: the prompt also told it "you speak only Polish, English and French" four hundred words further down. An instruction and a contradiction is not a wording problem. Emphasis was never the lever. Position was, and so was deleting the thing that fought it.

A gamepad, and six games on its face

Say "connect a pad" and a DualSense appears on the robot's face with the two buttons you have to hold pulsing, and it pairs. Then the sticks drive its head and its antenna, which is the only test worth having: a pad that bonds but whose sticks do nothing looks exactly like a working one from across the room.

Bluetooth lied to me three times getting there, each one worse than the last. pair returning quietly does not mean it paired. Connected: yes does not mean the pad is usable, and I had one paired and connected with no input device at all. And Paired: yes does not mean bonded, which is the one that explains the other two: the adapter had no pairing agent at all, so bluez brought the link up, reported success, stored no key, and then refused to attach the controller. The evidence for that exists in exactly one place, the bluetooth daemon's own log, and nowhere in the tool everybody uses.

Then games, because a pad with nothing to play is a diagnostic. There are six: pong, snake, bricks, space invaders, Galaga, and Atari's Star Wars trench run in wireframe. There is also a shelf that walks through them as covers until somebody picks one.

The number I am pleased with is not six. It is zero: all six games and the shelf were added without one line of the face renderer changing. A game is a module, one line of a catalogue, and one word in the brain's list of things it can do. That file already carries a branch per full-screen thing and must not grow one per game as well.

Two things I got wrong that only playing found. The paddle stick needed inverting, and reasoning from how the head is wired predicts the opposite. And the first paddles were too small, snake was drawn about three millimetres wide, and bricks read as dashes, because this panel is 170 dots per inch and sixteen pixels is two and a half millimetres. None of that is visible in a test. Render a frame, look at it, then deploy.

A web panel, and a robot that is genuinely asleep

All of that needed somewhere to be looked at, so there is a panel: every conversation, every voice it has heard and what it decided, the settings, the services, the logs, a microphone mute, and a power button. It is split in two, and the split is the whole design. An agent on the robot, because that is where the files and the services are. The panel itself on the server in the attic, because that keeps it out of the robot's Python entirely and it needs no ROS at all.

The best thing that came out of building it is that settings now survive a restart, which nothing did before. The worst thing it taught me had nothing to do with the panel. Our router rewrites the source address of every forwarded connection, so a request from the sofa and a request from the open internet arrive looking identical, and a rule that only allows the local network is decoration. It fails open and looks like it is working. I measured it by fetching the panel from outside the house and landing on its login page. It lives on an internal name now, where the guard is DNS and cannot be wrong.

And the robot now boots asleep and has to be addressed by name, which means nothing said in the room while it starts up is acted on. Asleep is genuinely asleep: the panel powers down after five minutes, the renderer stops drawing, and perception drops its subscription to the camera instead of merely slowing it. Measured, the face went from 70.8% of a core awake to 4.3% asleep and dark. A sleeping face is not a still face, which is why the middle of those numbers, 17.4% with the panel still lit, is so high.

One thing there is worth stealing. There is no backlight line back to the Pi on this panel, so it writes a power-down to the framebuffer's own blank control, and that survives the process dying. So the renderer turns the panel back on before it starts up, as a plain function against the file rather than a method on the object, because if the renderer is the thing that died then a method on it is exactly what will not run.

How I actually got there

Everything above describes the robot as it is now, which makes it sound like I designed it. I did not. Most of the current hardware is there because something before it failed, and two of the best features in the robot are accidents that came out of fixing something unrelated.

The power supply that took the amplifier with it

I started with a 5 V 5 A buck module. On paper that is plenty for a Pi 5. In practice it buckled under load: the Pi, the accelerator, the panel, the servos and the audio all pulling at once is not the same as the sum of their datasheet numbers, and the moment several of them peaked together the rail sagged.

So I replaced it with a 5.1 V 8 A module, which fixed it immediately. 5.1 V rather than 5 V is deliberate, incidentally: it buys back the drop across the wiring, and a Pi 5 that dips under 4.8 V starts throttling and complaining about undervoltage.

Then the new module turned out to want a minimum of 16 V on its input. The audio amplifier in the chassis was rated to 12 V maximum. There was no input voltage that satisfied both, so I took the amplifier out, and for a while the robot had no way to make a sound.

The microphone, and the speaker that came free with it

With no speaker to worry about, I moved to the other end of the chain, which was in worse shape anyway. The speech pipeline would not calibrate: whatever I set the thresholds to, recordings ended in the wrong place and transcription came back as nonsense.

The microphone was an ATR4697-USB, a conference microphone, the flat kind you put in the middle of a meeting table. It is a good microphone for that job and a bad one for this.

The failure was maddening because it presented as two opposite problems that turned out to be the same one. With automatic gain on and nothing to listen to, the device would ramp its own gain up until it clipped: a silent room reading a peak of −0.1 dBFS, the level meter pinned at full scale, and the wake word being fed garbage. So I turned automatic gain off, which stopped the saturation, and made the robot deaf instead: speech dropped from −33.5 dBFS to −52 dBFS, below the threshold the utterance gate uses, so the meter read a flat zero, every recording ended after 2.16 seconds of nothing, and Whisper answered near-silence with hallucinations.

I replaced it with the reSpeaker XVF3800, a four-microphone array with an XMOS XVF3800 doing beamforming and echo cancellation in hardware rather than in my code. It fixed the capture problem outright.

And then, reading the datasheet properly, I noticed it has a speaker output. With the array's own echo cancellation across it.

So the amplifier that the power supply had forced out of the chassis never needed replacing. There is no separate amplifier module needed: the speaker connects to the same board as the microphones.

And it is better than what came out. Because the cancellation is in the array, across its own output, I never have to mute the microphone while the robot talks, so you can interrupt it mid-sentence. That is the measurement from earlier in this article, and the whole reason talking to it feels like talking rather than like taking turns. A separate amplifier would have given me sound and none of that.

I would like to claim I planned it. I bought a microphone.

The whole chain, in order, because it is only obvious backwards:

1. The 5 A supply sags under load.
2. The 5.1 V 8 A replacement fixes that and demands 16 V in.
3. The 12 V amplifier cannot live on 16 V, so I take it out. No sound.
4. The speech pipeline will not calibrate, and the conference microphone is why.
5. I fit the XVF3800 to repair the ears.
6. It has a speaker output, with echo cancellation across it, so the amplifier is never missed and interruption works for free.

Not one of those steps was aimed at the thing it ended up fixing.

Heat, and the chassis I have not solved

I fitted the first fan after the thermal runaway described below, and it took idle from 63 to 67 °C down to 53 °C. That was enough right up until it was not: the accelerator, the new power module and the panel all put heat into a closed chassis, so I put a second fan in to keep the whole enclosure moving air rather than just the processor.

It is still not enough, and I want to be straight about that rather than leave a tidy ending here. The robot hard-reset again minutes after being fully assembled for the first time, under the heaviest load it has ever run: the accelerator inferencing, the camera, a network write, two API calls and speech, all at once. Open it up and it is fine. Closed, the fans cannot move the heat out. So it runs with its back off and will keep doing so until the chassis is redesigned for airflow, which is a modelling job rather than an electronics one and is the least interesting thing on the list. Every photograph of it working is a photograph of an unfinished enclosure.

Cooling has one non-obvious cost on a robot that listens. Fan noise is broadband and constant, and it sits directly on top of the threshold the microphone uses to decide somebody has stopped speaking. I have a measured note about exactly this: with a bench supply's fan running nearby, the noise floor stays above the gate, the gate never re-arms, and utterances stop ending when the person stops talking and instead run to the maximum length. It is intermittent and it tracks fan speed, which is a horrible thing to debug if you do not already suspect the fan.

Other things that did not survive

A shorter list, in no particular order:

• Sprites, for the face. Replaced by signed distance fields. Every expression used to be a file.
• Picovoice Porcupine, for the wake word. It was excellent, and it wanted a paid key per keyword. Now openWakeWord, which is free and trainable, and which I have not yet managed to train reproducibly.
• Whisper on the Pi. It runs. It does not run while the Pi is also drawing a face, tracking a person and holding a conversation. I moved transcription to the machine in the attic.
• YOLO on that same machine. Overtaken by the Hailo accelerator on the Pi. Sending 1280×720 frames over WiFi to detect a person in them was always going to lose to detecting them where they are captured.
• A self-hosted search engine, for finding the right Wikipedia article. I run searxng on the machine in the attic and went back to it recently to see whether a real search engine would rank articles better than Wikipedia's own search does. It cannot: searxng has no index, it queries public engines on your behalf, and 22 questions in under a minute got all three of mine either rate-limited or served a CAPTCHA. They had not recovered a quarter of an hour later. It fails quietly, too, returning zero results and a perfectly good 200.
• WAV, for the speech audio. A WAV header carries a length field written before the length is known, which is unanswerable over a stream. Raw PCM has no header to be wrong.
• The camera's full field of view. 68 degrees needs a sensor mode where the frame rate collapses to 7.9 fps. It runs at 48.6 degrees instead, and the robot turns its head.
• DALL·E 3, for the pictures it draws, which simply stopped existing on the account one day. I timed three replacements; the cheapest was also the slowest, which was not the answer I expected.
• The obvious model for telling voices apart. Newer, cheaper to run, and wrong about one pair of speakers in three. It is one of the failures below, because nothing in its output says so.
• Muting the microphone while the robot talks, which I built before measuring and then deleted, because the array's echo cancellation makes it unnecessary and being unable to interrupt it is worse than any problem muting solves.

The things that went wrong

Every project has these and most write-ups quietly leave them out. Six that cost me real time:

The robot kept hard-resetting, and I went into the rabbit hole. No log, nothing in the kernel ring buffer, and the throttling flags read clean afterwards, because the reset clears them. I convinced myself it was not load-related and wrote that down. It was thermal runaway. A fourth reset landed in the middle of a camera calibration; I fitted a fan, and idle went from 63–67 °C to 53 °C. The absence of evidence was itself the evidence.

The face rendered solid blocks instead of letters for a while, and every test passed. Rendering antialiased text with no background gives you a surface whose red channel is 255 everywhere inside the text box, with the actual coverage in the alpha channel. Read the red channel, and you draw a filled rectangle where the words should be. Every assertion I had was of the form "something was drawn", and something certainly was.

Printing was silently broken. The kernel numbers USB printers by enumeration order, the device moved from lp0 to lp1 across a reboot, and my code still named the old path. The error said "permission denied" on a device that was not there. There had been a stable symlink sitting unused the entire time.

A model that loaded, ran, and was useless. For telling voices apart I picked the obvious one: newer than the alternative, a third of the compute, and a config file in its own repository describing an ordinary training run. It loaded. It returned vectors of exactly the right shape and length. They responded sensibly when I changed the features. Every check I had said it was working.

Scored against forty real speakers it got one pair in three wrong. The model I had passed over got one in two hundred, through identical code on identical audio. Nothing about the first one's output looks wrong; you simply cannot tell from a vector whether it means anything.

Two things about finding that are worth more than the answer. I spent an hour convinced my own audio processing was at fault, and what settled it in one command was running two models through the same code, which separates "my features are wrong" from "this artefact is wrong". And I had been testing against six synthetic voices, which sit five times closer together than real people do. A weak test set does not fail. It makes a broken thing look marginal and a marginal thing look broken, and I blamed the test before I blamed the model.

The camera handed back an empty photograph, half the time. The robot would take a picture, write a file of zero bytes, log "photo saved", and then the brain would send that nothing to the model and get back an error about invalid image data, three steps away from a camera that was working perfectly.

It is the first frame after subscribing, and only that one. Eight captures in a row: four saw exactly one empty sample and then a good 214 kB frame on the very same reader. So it was not the camera, not the topic and not accumulated state; it was what a brand-new subscriber is handed on a topic whose frames are too big for one packet. The symptom is guarded now by throwing away anything suspiciously small and asking for a frame again.

The general lesson is: anything that subscribes, takes one frame, and unsubscribes should treat its first sample as suspect.

The common thread is that all six looked fine. This robot's characteristic failure is not a crash but silent success: a thing that reports it worked and did not. Which is why I now finish every change by looking at the artefact: the pixels on the panel, the paper out of the printer, the actual sound in the room. Not the test result.

Where it is now, and what is next

It wakes when called, follows you with its eyes, knows which of us is talking and answers in that person's language, tells you how warm the house is and turns the lights off, looks things up and prints them, plays six games, and introduces itself to visitors without me in the room. Seventeen services on the robot and two in the attic, a few thousand lines of Python, and about a tenth of a cent per conversation.

It also stays exactly where I put it. The chassis will have wheels, and the motors and the ESP32 controller will sit in it. At this point still unwired, so for the moment this is a robot that turns its head rather than one that goes anywhere. Wiring the drivetrain is the next real milestone and the one everything else is queued behind: navigation and SLAM both need wheel odometry, and there is no odometry without turning wheels. The arm is at the same stage, one printed joint and an intention.

Ahead of the wheels, in the order I would take them:

1. Its own wake word. It answers to a stock one, "Hey Marvin," but it is growing on me. Training a custom one works and then does not reproduce, and I will not deploy something I cannot rebuild.
2. Somewhere of its own to look things up, on the server in the attic, so it stops depending on the network for everything. The integration point already exists and costs nothing per question. The open question is the only one that matters and it is not technical: what should be in it. Household documents and appliance manuals, the children's schoolbooks, its own past conversations, or an offline encyclopaedia are four very different machines.
3. A chassis that can be closed. Unglamorous, and it is the thing standing between this and a robot rather than a project.
4. Then the wheels, and everything that has been waiting behind them.

The vertical gaze bug is still there too, and I have left it there on purpose while writing this, because it is the best example I have of something that looks like a one-character fix and is not.