RethinkKNX: a house that keeps a log of itself

At twenty to eleven this morning the blinds on the top floor came down, and nobody asked them to. Four hours earlier something had worked out that the sun would be on that glass by eleven, that the room would reach 27.1 degrees if it did nothing, and that closing the blinds would hold it to 26.8. It wrote that down, with the number it would have got for doing nothing, and waited until 10:35 to act on it.

That is the house I have been building since December 2016. It is a KNX installation, which is an ordinary and rather boring thing to have in a wall, plus about nine years of software that tries to make it less boring. This is how it fits together, what it gets right, and the part where it confidently concluded that sunlight has no effect on the temperature of a room.

The short version

Everything below is how each of these was built and what it cost. Stop here if that is all you wanted.

It does.

• Every telegram on the bus becomes a row in a database, in and out, so the house can be asked about last Tuesday.
• Seven rooms run as separate processes, each watching only its own devices, and each able to be restarted without the others noticing.
• It forecasts each room's next 24 hours, simulates a handful of ways to keep it in its comfort band, prices them by how much they would disturb whoever is in the room, and picks the cheapest one that works. It logs the decision it would have made against the one it beat.
• It forecasts the solar production, the household draw and the battery, all calibrated from this installation's own history rather than from anybody's datasheet.
• It drives an iPhone app, an iPad app, a browser, an e-paper panel on the wall, a small colour display, and a fleet of ESP8266 and ESP32 boards that were never going to speak KNX.
• It exposes the house to a robot, carefully.

It does not.

• It does not act on its own cooling decisions. The planner has run in shadow mode for weeks: it decides, it logs, and it changes nothing. Switching that over is a one-line config change and I have not made it, for a reason I will get to.
• The thermal model is still marked uncalibrated for most rooms, and the reason is not that the code is wrong.
• There is no high availability of any kind. The server is a box in the attic. If it dies, the house reverts to being a KNX installation, which is the one genuinely good thing about building on top of a standard that works without you.

Where it came from, and what was wrong with it

KNX is a field bus. Two wires run round the house and every switch, relay, blind actuator, thermostat and meter hangs off them. Each one has a group address, and this house has 344 of them across 29 datapoint types. Press a switch and a telegram goes out; a relay somewhere hears its address and closes. No server is involved and none is wanted. It has worked for decades and it will keep working the day I finally break something.

The problem with it is the sentence I just wrote. A telegram goes out and a relay closes. That is the entire transaction. The bus has no memory. Ask KNX how warm the bedroom was on Tuesday night and there is nobody to ask, because Tuesday night was a series of voltage changes on a pair of wires and they are gone.

My first attempt at fixing that was a Raspberry Pi under the stairs running eibd, which I wrote about back in 2014. It worked. It also did nothing except let me send telegrams from a script, which is a remote control with extra steps. The daemon has since been replaced, and that turned out to be a story of its own, mostly about how little had to change.

The interesting decision came later, and it was this: stop treating the bus as something to command, and start treating it as something to record.

The one idea the whole thing rests on

Exactly two processes are allowed to touch the wire. One listens to everything the bus says and writes each telegram into a table. The other watches that same table for rows addressed outward and puts them on the wire. Neither of them knows what a light is.

Everything else in the house - every room, the cooling planner, the meters, the web server, the phones - reads a changefeed on that table and writes rows back into it. RethinkDB does the heavy lifting here: you subscribe to a query and it pushes changes at you, so a room worker is a loop over the events matching its own addresses, and nothing polls.

Three things follow from that, and they are the reason the rest of this article is possible.

1. The house is answerable. Every telegram for years, with the value decoded through its datapoint type and, where it was the software that sent it, the reason. This is what makes the thermal model in the next section a thing you can fit rather than a thing you can guess.
2. Nothing needs the bus to be tested. The whole unit suite runs without a database, let alone a KNX interface. There are 54 test files, and the test harness fast-fails any accidental real connection with a labelled error rather than letting it hang for two minutes retrying.
3. New things are cheap. An LED strip that speaks MQTT and a blind that speaks KNX are, from the point of view of a room, the same kind of object: something with an address that produces rows.

Parts and rooms

A part is one device. It knows its group addresses, it has an update_internal_state that gets handed matching events, and it emits telegrams by writing rows. There are 39 files in that directory and 16 classes wired into rooms across 14 categories: lights, blinds, windows, thermostats, sockets, sensors, an air purifier, air conditioning, RGB strips, a robot vacuum, remotes, and a few things that are really appliances wearing a part costume.

A room is a container that opens one changefeed over all of its members' addresses and holds the automation. Seven rooms in the config, nine room modules, because two of the modules are staircases that behave like rooms without being ones.

The piece I use most is the least clever. A virtual part is a name, a group address and a datapoint type picked from a dropdown, created from the web editor. It shows up in the room, on the phone and on the panel, and it is picked up live: the room worker gets a synthetic reload event and re-subscribes without a restart. Most days, adding something to the house is now typing an address into a form.

There is a scar on that screen worth pointing at. The parts table used to be keyed by name alone, so two rooms with a part called window were, as far as the database was concerned, one part. Renaming one clobbered the other. The fix was a one-time migration to a {name}_{room} key, and the lesson is the ordinary one: the bug was not in the code that broke, it was in a primary key chosen in an afternoon four years earlier.

What is actually running

Twenty-nine processes on the server, under supervisor, plus two that cannot live there.

Those last two are a good illustration of why hard-won knowledge belongs in writing. The sensor process reads a temperature and humidity chip over I2C, which means it has to run on hardware physically wired to that chip - a Raspberry Pi by the front door, not the server. That Pi runs Python 3.5 and a checkout from 2022 that cannot be updated, because current code uses f-strings and the interpreter predates them.

Which is how a CO2 chart froze at 769 ppm for weeks. Every deploy went out cleanly, every process restarted, and the one that mattered was on a different machine and had not been touched. There was nothing to debug in the code. It is now the first line of the deployment notes.

The cooling planner, which is where it stops being a hobby

The rest of this system is plumbing, and good plumbing is invisible. This part is the one I would show somebody.

The problem: this house gets hot in summer, and it has four ways to do something about it. Close the blinds. Open the windows. Run the air conditioning. Or wait. Each has a different speed, a different cost, and a different level of rudeness, and the correct choice at two in the afternoon is the wrong one at two in the morning.

The naive version of this existed for years and was one line: if the room is over 27 degrees, switch the air conditioning on. It knew nothing about the sun, the battery, the time, or whether anybody was asleep. It has been deleted.

It plans units, not rooms

The first thing that had to change was the noun. The salon and the dining room are two rooms in the config, but they share one temperature sensor, one air conditioner and one blind. Planned separately they produced two identical decisions every tick, two telegrams on the same address, two identical alerts, and - the good bit - a salon plan that kept deciding to close a blind the salon does not have.

So the planner works on planning units. The top floor is one unit with two member rooms folded into a single state: temperature from the member that has a sensor, blind position from the member that has a blind, and a window counted open if any member's is. The history stays per room, because that is a measurement and measurements should not be averaged before you know what for.

What a move costs

Every five minutes each unit gets a 24-hour forecast, a set of candidate plans, and a simulation of each one against a thermal model of the room. The cheapest plan that keeps the predicted peak inside the comfort band wins. All of which is unremarkable until you ask what cheapest means.

It is not kilowatt-hours. The table is:

BASE_COST = {
    "shade": 1,
    "window": 1,
    "ac": 6,
}

OCCUPIED_SHADE_SURCHARGE = 1
OCCUPIED_WINDOW_SURCHARGE = 1
QUIET_HOURS_WINDOW_SURCHARGE = 5

A blind costs 1 and the air conditioning costs 6, which is not a claim about electricity. It is an ordering: exhaust the free options before spending anything. Then the surcharges, which are the actual idea. A move that somebody will notice costs more, and a window is the one actuator in the house that can wake a sleeping person.

Add it up for a hot night in an occupied bedroom. Opening a window is 1 + 1 + 5 = 7. Running the air conditioning is 6. So the house buys electricity rather than waking you, and it does that because of three integers rather than because of a special case. When somebody is away, no surcharge applies at all and the same code happily throws the windows open instead.

Speed is deliberately not in that table. It does not need to be: a plan that is too slow simply fails to hold the band in simulation and loses on the merits. A closed blind is worth about 0.06 degrees an hour and an open window about 0.075, and neither is fast enough to start late, which is why the planner acts a full degree before the ceiling rather than at it.

The air conditioning has its own gate on top of all this, and it is about the solar array rather than about comfort. It may only start when it is warmer outside than in, outside a 21:00 to 08:00 window, with the boiler not heating, with the panels making over a kilowatt and the battery above 35 percent. On this installation that confines it to roughly 08:00 to 18:00 on a sunny day, which is a polite way of saying it only runs on someone else's money.

Two details in that rule I would defend. A stale inverter reading refuses the start, because no data cannot be allowed to mean the sun is paying. A stale boiler reading does not block, because an unrelated MQTT bridge falling over must not disable cooling in August. And it is a start gate only: an air conditioner somebody switched on by hand at 23:00 keeps running, because the house is not the boss of anyone.

That screen is the honest one. Auto actions: off. The planner has been deciding for weeks and actuating nothing. Every decision goes into a log with the counterfactual it beat, and the reason I have left it that way is in the next section.

The week the model decided sunlight does not exist

The simulation is a small physical model of a room: it leaks heat to outside at some rate, gains it from sun through the glass, loses it through an open window, and the air conditioning pulls it toward a setpoint. Five coefficients. They are refitted every night from the room's own recorded history, which is exactly the thing the events table was for.

One morning I looked at the fitted coefficients for the top floor and the solar term was zero. Not small. Zero. Which means that in every simulated plan, closing the blinds did nothing at all, and the planner had been solemnly comparing shading strategies that its own model believed were no-ops.

The bug was not in the fitting code. It was in the house. I went back through the history and found this: of 140 daylight snapshots with the blind down, 71 percent had the air conditioning running. Of 487 with the blind up, 12 percent did. I only ever close that blind on the days I also run the AC. Solar gain and AC cooling are therefore almost perfectly confounded across nearly the whole record, and no fitting procedure on earth can separate two things that always happen together.

There was one clean slice: daylight hours with the AC off. Blind up, the room warms at +0.058 degrees an hour (n=425). Blind down, it cools at 0.030 (n=40). So the blind obviously works, and the data as a whole simply cannot see it.

The fix is a floor, not a smarter regression. Below a certain value the solar coefficient is held at a prior and the other terms are refitted around it. The prior is deliberately set to about half of what that clean slice suggests, because I would rather under-credit the blind than have the model start inventing sunshine. On a fortnight of real data it moves the error from 0.0793 to 0.0800 - very slightly worse on paper - and makes closing the blind worth 0.057 degrees an hour instead of nothing. I took the worse number.

I also capped the free intercept, because a term with no physical meaning and no constraint is exactly where a real effect goes to hide.

And the one it found that I did not know

The reverse also happened, and it was better. Night ventilation kept losing to plans that made no sense, and the reason was that the model treated an open window as an open window regardless of the blind in front of it. These are solid insulated blinds. So I went to the history again, this time for one child's bedroom at night with outside at least three degrees cooler:

• Window open, blind up: cools at 0.0484 degrees an hour per degree of difference (n=36).
• Window open, blind down: 0.0079 (n=372).

Six times. And with the blind down, opening the window at all barely registers: 0.0066 becomes 0.0079. A closed blind does not just block sun, it blocks the air, and I had been modelling the two as independent for months.

Fixing it took that room's fitted time constant from 520 hours - which the sanity check was correctly rejecting as nonsense - to 277, accepted, and more than doubled its ventilation coefficient. It also changed the behaviour: a ventilation plan now raises the blind too, wherever the blind is free to move. That replaced a whole separate module I had written to special-case night purging, which had only ever existed because blind position used to multiply out of the night-time model and so could never win on its own.

This is the argument for keeping every reading. Both of those findings are a database query. Neither is available to a system that only knows what the house is doing right now, and I would not have believed either of them from intuition. I would have guessed the blind mattered less than it does for sun, and far less than it does for air.

Which brings me back to why it is still in shadow mode. That log is the argument for switching it on and the argument against, at the same time. Most of the entries are right. Some are right for reasons I have not verified. Most rooms still say uncalibrated, because three days of clean samples is the bar and this summer keeps interrupting itself. Reading a fortnight of decisions I did not have to live with is cheap; reading them after the house woke somebody up is not.

The one thing that acts anyway

There is a single exception, and it is deliberate. A watchdog measures how long the air conditioning has actually been running, warns once past fourteen hours, and past eighteen - with the room already comfortably below target - switches it off. That runs regardless of whether autopilot is enabled, because a unit nobody turned off is a fault, and a fault is not a preference.

It exists because of a 23-hour run. The old code tracked runtime on the AC object itself, in a field that was only ever assigned inside an if self.ir: branch. This unit has no infrared transmitter of its own, so the branch never ran, the field stayed empty, and the safety check compared against nothing for as long as it had existed. The watchdog now measures from the recorded history instead - the same snapshots the model is fitted from - because that is a fact about the house rather than a fact about an object's memory.

Energy, and the refusal to draw a line

There is a solar array on the roof, a battery, and an inverter that gets polled every ten seconds. The interesting part is not the current reading, it is the next 24 hours.

Nothing in that pipeline knows the panel rating. Match what the roof produced against the irradiance it was produced under, and the conversion factor falls out of the arithmetic, with the shading, the dirt, the inverter's real efficiency and the angle of the roof already in it. The same trick gives the battery's kilowatt-hours per percent in each direction, its actual power limits and the reserve it will not go below, none of which match the datasheet and all of which are true of this battery.

The household draw is the part I like. It is a weighted mean over the last seven days, by time of day in fifteen-minute buckets, with each past day weighted by how closely its sky matches the sky that is coming. A cloudy Tuesday is a better guide to a cloudy tomorrow than yesterday was, and the house has enough history to know which Tuesday that was.

And then the bit I would argue about with anybody. The forecast is drawn as a widening fan, not a line.

Every predicted point carries an inner and an outer band, pinched shut at now and spreading with how far ahead it is. It would be trivial to draw the mean as a clean confident curve and it would look considerably better. It would also be a lie told in a nice typeface, and I have to make decisions off this chart - whether the washing machine can wait for the sun - where the honest answer is often the model does not know. A flat green band sitting on the reserve floor overnight is the chart saying the grid takes over at four in the morning, and I would rather see that with its uncertainty attached.

One number on that page was wrong for months in a way worth confessing. The inverter publishes a register for energy imported today, and on this unit it reads zero, always. Rather than derive the figure by integrating instantaneous power and hoping, today's grid import is now a difference between two readings of the smart meter's lifetime import counter against a baseline stamped at midnight. The dead register is still published, purely so an older build of the phone app can still decode the payload.

Everything that reads it

All of the above would be a cron job with opinions if there were nothing to look at. There are four ways in, and everything hangs off one of them.

The apps, and the contract that stops me breaking them

There is an iPhone app, czeska, and an iPad one, CzeskaXL, in three columns. Both are Swift and UIKit, both take REST for state and Socket.IO for the live view.

The failure mode with a private API and a native client is specific and stupid: I rename a field on a Tuesday, the app's decoder throws on a type it did not expect, and the first I know about it is a red X where a switch should be. Nothing logs. Nothing alerts. The phone just quietly stops working, in a way that only shows up when somebody reaches for a light.

So there is a checker that parses the Swift models the apps decode into, extracts the contract, and validates real API payloads against it, mirroring the type rules Swift's JSON decoder actually applies. It runs offline against committed fixtures, it runs in the test suite, and a git hook refuses the push when the two disagree. It is the single highest-value hundred lines in the project, and it exists because everything else about a schema mismatch is invisible.

The desk panel

The e-paper display is a CrowPanel ESP32-S3, and the division of labour is the whole trick. The panel does not compose anything. It asks the server for a screen; the server renders an HTML template with a headless browser, dithers the result to one bit per pixel, and returns 26,928 bytes, which is 792 by 272 divided by eight. The panel draws the bytes.

Which means changing what the wall says is editing a Jinja template on the server, not reflashing anything. Layout, typography and what counts as important all live where they can be changed in a minute. The microcontroller's job is to be a picture frame with wifi, and it is very good at that.

The same reasoning covers the small colour display that shows notifications and the ESP boards scattered around: an LED strip in the bedroom, a temperature and CO2 hub, a motion and light sensor in the corridor, a PWM dimmer for the beacons, and a door controller that ends in a KNX relay. They all speak MQTT, one bridge process folds MQTT into the same table as everything else, and from a room's point of view a cheap ESP8266 and a certified KNX actuator are the same kind of thing.

Letting a robot in

Kompot is the robot on my desk. It answers questions, and one of the things it can be asked about is the house: how warm a room is, whether anybody is home, what is on the calendar, what is playing. It can also change things.

That last sentence is the whole design problem, and it is not the one you would expect. The server it talks to never leaves the local network, it wants a bearer token, and it checks the Host header against an allowlist so a web page cannot rebind DNS and walk in. Fine. But the threat that shaped the rules is not an actual blackhat attacker at all. It is written in the source, and I will let it speak for itself:

it prevents a small attacker - it is a eight year old asking the robot to open her bedroom window in February, and the robot cheerfully doing it.

A token authenticates the robot. It says nothing about who is standing in front of it. So reads are free, and every write carries the name of whoever the robot recognised by voice; writes from anybody not on the adult list are refused, and so is anything at all during quiet hours. The refusal is a sentence the robot says out loud, not an error.

There are two of these servers, one on the internet and one that is not, and they share a single set of tool definitions on purpose. Two definitions would eventually answer one thing to a chat client and another to the robot, and the day they diverge is the day the safer of the two stops being the one that matters.

One more detail I would keep. Every reading the robot reports carries its age, because a robot that says the bedroom is 19 degrees from a week-old row is worse than one that says it does not know. Confidently stating stale data is the failure mode that makes people stop trusting a system entirely, and it costs one timestamp to avoid.

What it adds up to

Nine years, 1,401 commits, and about 69,000 lines of Python of which 15,000 are tests. Fifty-eight tables, 180 routes, 344 group addresses, twenty-nine processes and two more on a Pi that cannot be upgraded. One house.

The decision I would make again without hesitating is the boring one at the top: two processes touch the wire and everything else talks to a database. Every good thing in this article is downstream of it. The thermal model is fittable because the readings were kept. The blind discovery is a query. The apps, the panel, the ESP boards and the robot were all added without going near the bus, because none of them knows the bus exists.

The decision I got wrong is smaller and more embarrassing: I let the documentation describe the system instead of the system's reasons. Which is how a diagram in this repository still shows a daemon that was replaced last November, and how a safety check sat there for years comparing against a field that was never assigned. Both were obvious the moment somebody looked. Neither was going to announce itself.

Next, in the order I would take it: get enough clean days to calibrate the rooms that still say uncalibrated, then turn autopilot on for one unit and one summer and see whether I trust it. Everything else is waiting behind that, because a planner that only ever writes to a log is a very elaborate way of being right in private.