I have microphones outside that listen for birds. A neural net running on a little server chews on the audio around the clock, and every time it’s confident it heard a species, it posts the result to my home dashboard. Common Chiffchaff at 94% confidence. Great Tit. Blackcap. The dashboard is where I glance to see what’s been singing.
For a while, the dashboard said nothing. Every bird sensor sat at unknown. Not zero, not stale, just the gray non-value Home Assistant shows for a sensor that has never once been written. The obvious reading was that detections had stopped reaching the dashboard. The obvious reading was wrong.
The birds were there the whole time. Every detection was making the full trip — off the microphone, through the detector, out onto the network, into Home Assistant — and getting turned away at the very last step, on the doorstep, for a reason that took me embarrassingly long to see.
How the pieces fit together
Here’s the shape of the system, glossed for anyone who doesn’t run one. BirdNET-Go is the detector: it takes audio streams, runs each one through a bird-identification model, and publishes every detection as a small JSON message to an MQTT broker. MQTT is the lightweight publish-subscribe bus most home-automation gear speaks. Home Assistant, my home dashboard, subscribes to that bus.
The clever part is that I never defined the sensors by hand. MQTT has a discovery convention: the publisher announces “here is a sensor, here is its name, here is the little template that pulls its value out of my messages,” and Home Assistant auto-creates the entity. The detector announces one sensor per audio source. Home Assistant builds them. In principle you plug it in and birds appear.
The log had the answer the whole time
For longer than I’d like to admit, I let it ride, assuming the detector had fallen over and I’d get to it. What finally moved me was running out of guesses. When something in this chain breaks, the instinct is to start at the ends — is the detector running? is the broker up? — and work inward. That’s slow. The fastest thing I did, and the thing I’d do first every time now, was read Home Assistant’s own log.
It was not subtle. For every single detection, a line like this:
Value error while updating state of sensor ... it has the non-numeric value: 'unknown'That one line reframed the whole problem. The detections were arriving. Home Assistant was receiving them, parsing them, and matching them to sensors. Then it looked at the value it had extracted, found the string unknown where it wanted a number, and refused to store it. Nothing upstream was broken. The broker, the network, the discovery handshake — all healthy. The failure was a value, and the log handed me the exact rejected value on a plate.
A fallback that wrote a landmine
So where did the string unknown come from? This is where it gets properly annoying.
The detector publishes every source’s detections onto one shared topic. To split them back apart, each auto-created sensor carries a template with a filter: if this message is from my source, compute the confidence; otherwise, fall back. Reasonable. The trouble is what it fell back to. The template read, in effect: if the source matches, return the number, else return this.state.
this.state is the sensor’s own current value. The intent is obviously “if this message isn’t for me, leave my value alone.” That’s not what it does. In the templating language, this.state renders the literal text of the current state. And the current state of a sensor that has never been written is the literal string unknown. So the fallback doesn’t leave the value alone. It actively writes the word unknown into a slot that demands a number, and Home Assistant, correctly, throws it out. The template that was supposed to guard the sensor was poisoning it.
There’s a documented right way to do this, and it runs on the opposite instinct: instead of returning something on the no-match branch, return nothing at all. An empty result tells Home Assistant to skip the update silently. “Do nothing” is an outcome the platform understands; “the string unknown” is not.
A second problem sat underneath the first and made it worse. Those per-source filters bake in the source’s ID at the moment the sensor is announced. My detector, across a handful of restarts, reassigned its source IDs. New identifiers, same microphones. But the old announcements are retained messages, the kind the broker holds onto and replays to anyone who reconnects. Nobody had told the broker to forget them. So even the sensors whose templates could have matched were now filtering against IDs that no longer existed, and every one of them fell through to the poisoned fallback.
Both causes had to be present to blank the dashboard, and on the day of the outage both were. Every sensor had drifted off its old source ID, so even a fallback that did nothing would have left the board just as empty — only quietly, with no error log to tip me off. So the fallback isn’t the bigger cause; it’s the one worth fixing, and that’s a different thing. Fixing it is permanent: a no-match branch that stood aside would defuse every future drift in a single change. Clearing the retained messages only cures the churn in front of me — the sources will drift again, and I’ll be right back here. The drift keeps lighting the fuse. The fallback is why there’s a fuse to light at all, and it’s the end I can actually remove.
Clearing the cache the broker wouldn’t forget
The fix was less satisfying than the diagnosis. I deleted the stale sensors, cleared the retained announcements the broker was hoarding, and let the detector announce itself again from scratch against its current source IDs. Fresh templates, matching IDs, real numbers landing in real sensors. The dashboard lit up with birds inside a minute.
I want to be honest that this is a patch, not a cure. The else this.state fallback still lives in the detector’s code. Any future drift between the live sources and the retained announcements will resurrect the exact same symptom, and I’ll get to rediscover it, unless the upstream template changes to the do-nothing form. Filing that upstream is on my list. Until then, at least I know where the body is buried.
What I want to keep
A few things came out of this that I want to hold onto.
“No data” is almost never a single switch. A pipeline like this has half a dozen segments — capture, detect, publish, deliver, parse, store — and “nothing’s showing up” can mean a break in any of them. Don’t debug it as a binary; find the segment.
Reach first for whatever component logged what it rejected. That single move collapses the “which segment failed” question, because the thing that refused the value usually names it. It won’t hand you the root cause — I still had to trace where that unknown was coming from — but it tells you where to stop guessing. Here it was the dashboard’s own error log, and it should have been the first place I looked.
A fallback should fail toward doing nothing. The whole bug is a fallback that chose to write something — a plausible-looking something — instead of standing aside. “Leave it unchanged” and “write the current value back” sound identical right up until the current value is a sentinel the next stage can’t swallow. When in doubt, the safe default is the empty result, not the clever one. The catch is that a fallback which does nothing also reports nothing: the same silence that would have saved me here is the silence that erases the error log that actually cracked the case. Failing toward doing nothing is only safe when something is watching for the nothing — which is what the alarm below is for. Skip that, and you’ve only traded a loud wrong answer for a silent one.
Systems that configure themselves can misconfigure themselves silently. Auto-discovery is wonderful until the announcement and the reality drift apart. Nothing anywhere says “your saved sensor definitions describe a world that no longer exists.” The retained message just sits there, quietly wrong, and the only tell is that nothing updates. If a system builds itself from a cache, you need a way to blow that cache away and rebuild — and you need to remember the cache is there at all.
The gap that let this hide for so long was the absence of an alarm. One sensor stuck on unknown is invisible. A log steadily filling with the same error line is also invisible, right up until a human happens to read it. That asymmetry — loud in the logs, silent everywhere a person actually looks — is exactly what monitoring is for. I’m adding an alert on that class of error so the next occurrence pages me instead of waiting for me to notice the birds have gone quiet.
The birds, of course, were never quiet. They were singing into a microphone the entire time, detected and published and delivered, stacking up rejection after little rejection with every fresh detection. The whole failure lived in one word, on one fallback branch, choosing to say unknown when it should have said nothing at all. Next time the dashboard goes blank, I’m reading the log before I believe it.