A snapshot from May 2026. The tended, current-state version lives in the garden: home-assistant-architecture.
The house runs on about 40 YAML packages, a handful of custom blueprints, a bespoke voice pipeline, and a calendar integration that turns family schedules into automated routines. Blog/Posts/My AI-native Home(lab) covers the wider Sense–Plan–Act architecture and the two-plane split between deterministic rules and the AI that writes them. This post is the Home Assistant deep-dive: how the packages are organized, what the blueprints abstract, and where the deterministic plane earns its boring reliability.
Infrastructure
Home Assistant OS runs on a Proxmox VE hypervisor — one node in a three-node cluster of Lenovo M75q Gen 2 machines with 32 GB RAM each. The cluster survives a single-node failure without split-brain. ZFS replication across nodes handles data durability, and Proxmox Backup Server covers 3-2-1 backups.
Proxmox also hosts LXC containers and VMs for supporting services: a Debian VM running Docker Compose stacks (BirdNET-Go, VictoriaLogs, Grafana, and others), a Claude Code agent LXC for task automation, and a Google Photos sync container for the wall panel slideshow. Syncthing keeps configuration files synchronized across machines — I edit HA packages from my Mac and the changes land on the HA instance within seconds. It also gives me a natural disaster recovery path without a separate backup ritual.
Tailscale provides remote access. Caddy handles reverse proxy with SSL termination. Split-horizon DNS via the UDM-Pro’s dnsmasq routes internal requests locally (a Technitium DNS migration is planned).
Configuration organization
Everything lives in HA’s package system. Instead of one massive configuration.yaml, around 40 YAML files in a packages/ directory each scope to a functional domain: lights.yaml, climate.yaml, presence.yaml, sleep.yaml, and so on. Each file is self-contained — helpers, automations, and scripts for that domain live together. Room-specific overrides get their own files when they diverge enough from the defaults.
On top of packages, custom blueprints handle the patterns that repeat across rooms: presence-activated lights, air purifier management, HVAC control, alert state machines, reminders, and person presence tracking. Adding a new room to the lighting system means instantiating a blueprint with a few parameters, not copying and modifying a 50-line automation. Every state trigger excludes transitions from unknown and unavailable — template reloads cause sensors to briefly pass through those states, and without guards, every automation fires spuriously on reload.
Network and cameras
The hardware is a UniFi-first stack — one vendor for routing, switching, Wi-Fi, access control, and video.
VLANs segment IoT, cameras, kids’ devices, guests, and trusted clients. Content filtering at the firewall blocks ads, trackers, and adult content for the kids’ VLAN (DNS-based filtering is bypassable; firewall-level is not). CEF syslog ships UniFi events to VictoriaLogs, where they join HA logs and container logs in a single LogsQL query surface.
The front door is on UniFi Access: an Enterprise Access Hub in the rack, a G3 Reader Pro at the door (NFC + PIN + face recognition), and a Shelly relay wired to the existing electric strike. Access events stream into Home Assistant — a recognized face can disarm the alarm, turn on the entryway lights, and announce who arrived. An unknown face captures a snapshot, posts it to Discord, and arms a 60-second alert window.
Cameras are on UniFi Protect, recorded to a UniFi NVR. Smart-detect events (person, vehicle, package, animal) flow into HA as binary sensors and become triggers in the same automation engine that runs the lighting. The wall panel embeds RTSP streams directly for sub-second latency rather than going through HA’s image proxy.
Package detection (LLM vision)
A camera watches the package box on the front step. The detection pipeline is not a trained image model — it’s an LLM vision call. Two triggers feed the same automation: motion on the package-box camera, and a proactive periodic scan (every 30 minutes during business hours, when the last check was more than 4 hours ago). Both capture a high-resolution frame, send it to Gemini Vision via ai_task.google_ai_task, and set input_boolean.package_detected based on the result. The alert blueprint handles notification, and a re-check loop runs every 30 minutes until the package is cleared.
The proactive periodic scan exists because IR motion sensors miss packages dropped silently. The re-check branch handles the inverse case — clearing the detection automatically once the package has been brought inside. Both branches share the same AI call and the same alert state machine.
Bird identification
The garden cameras have microphones. BirdNET-Go runs as a Docker container on the Debian VM, reading the cameras’ RTSP audio streams directly — no dedicated outdoor microphone needed. The TFLite BirdNET model identifies 6,500+ species in real time; detections publish over MQTT with HA auto-discovery, creating sensors for last species, confidence, scientific name, and sound level per camera.
A daily template sensor aggregates interesting detections (rare species, high-confidence matches) and feeds them into the morning briefing — “You had a great spotted woodpecker in the garden yesterday afternoon.”
Presence and people
Room-level presence is the foundation for every targeted automation. It fuses three signals:
- GPS (companion-app device trackers) for is-home / is-away
- Bermuda BLE for room-level localization inside the house — every phone is a beacon and every ESPHome-flashed Shelly Plus Plug S is a receiver, no separate beacon network needed. The same plugs do double duty as energy monitors that detect washing-machine and dishwasher cycle completion via current draw.
- Calendar for “where should this person be right now?” —
sensor.person_*is calendar-aware, so a kid’s school event during school hours sets virtual coordinates and a synthetic “away” state even if the BLE tracker says they’re nearby
Each person has a context_entity that exposes their current room. The announce package reads this to route TTS to the right speaker. The climate blueprints read it to drive per-room setpoints. The lighting blueprints read it to know whether to turn on.
The presence system feeds the alarm (Alarmo arms on departure, disarms on arrival), door locks, and announcement routing.
Circadian lighting
Lighting is presence-activated and time-of-day-aware. The presence_light blueprint runs in every room with inputs for presence sensors, indoor and outdoor lux sensors, main and accent lights, and adaptive brightness/temperature driven by the circadian phase.
A central input_select.circadian_phase tracks the current phase: morning, day, evening, wind-down, and night. A single automation manages transitions via a priority cascade — sleep state takes priority, then wind-down time, then evening (past a configured time AND sun below horizon), then a post-wake morning window, then the default day phase.
Why time + sun, not just sun: the earliest sunset in Munich is around 16:21 in December. Letting indoor lighting follow the sun puts the house in evening mode while everyone is still working — actively bad for SAD management. Indoor lights stay in day until input_datetime.evening; outdoor accents follow the sun. The circadian phase is the dial that lets these decouple.
The phase also drives announcements (good morning vs. wind-down vs. good night), accent lighting on/off, and the bed sensor’s nap-vs-bedtime distinction. Manual override is supported — the phase is an input_select, so any UI, voice, or automation can force it; the cascade resumes at the next natural transition.
Climate and air quality
Each room with a Daikin unit has its own HVAC manager blueprint instance. The blueprint factors in target temperature, humidity, window sensor state, outdoor conditions, and occupancy. Rate-limiting guards prevent redundant API calls to the Daikin units — without them, the system would saturate the embedded web servers with around 100 requests per hour, causing periodic unavailability. The blueprint also has DND awareness (don’t change the setpoint while someone’s asleep) and a vacation mode that lowers setpoints for extended absences, driven by the calendar integration.
Air quality is its own concern:
- FTLab RadonEye RD200 Plus 2 in the basement triggers a critical alert at the 100 Bq/m³ threshold and overrides HVAC ventilation to bring fresh air through
- CO2 sensors in the bedrooms feed sleep-quality metrics and prompt a window-open suggestion when levels climb at night
- PM2.5 / VOC from the same sensors trigger air purifier activation per room
- Pollen (external API) gates whether the bedroom shutters open in the morning — high-pollen days keep them closed until purifier runtime catches up
Air purifiers follow the same blueprint pattern. Fan speed is based on PM2.5 readings, adjusting down during sleep hours or when Do Not Disturb is active.
Sleep orchestration
Sleep is a master switch (input_boolean.sleep) that flips the house into night mode. The sleep script runs a sequenced routine:
- Shutters close in a staggered sequence (1s apart to avoid RF collision on the one-way Becker transceiver)
- Lights turn off (filtered — reading lights and specific accent segments are excluded so bedtime reading is undisturbed)
- Front door locks and Alarmo arms to “Night”
- Wake-up alarm evaluation runs and the result is announced in the good night briefing
- Bed warmer prompt offers to activate heated underblankets, gated so it only fires once per wind-down
Per-room sleep entities allow one person to trigger the routine for their room when they head up early. A Withings sleep mat under the mattress drives heart rate, breathing, and sleep-stage transitions into the sleep package — feeding bed-warmer activation and bedtime announcements. The primary sleep trigger is still manual.
Do Not Disturb composes manual toggles with sleep state. Each room has its own DND boolean, and a template binary sensor combines manual_toggle OR (room_asleep AND elapsed >= delay_minutes). The delay prevents an accidental sleep toggle from instantly silencing a room. The announce package and climate blueprints both consume these sensors — DND suppresses both noise and HVAC adjustments that might wake someone.
The alert stack
Home Assistant runs a purpose-built alert system for household-level events. The architecture is one blueprint, instantiated dozens of times, running each instance through a three-state machine: Active (fires notifications at a configured repeat interval), Acknowledged (silences notifications but keeps the alert open), Resolved (fires a final all-clear notification).
A second blueprint, alert_numeric.yaml, creates threshold-based binary sensors with hysteresis — turning “radon level vs. threshold” into a binary alert sensor. The same pattern covers humidity, temperature, and battery.
Each alert instance can declare watched_entities — sensors that are themselves health-checked. If a leak sensor goes unavailable, the blueprint fires a separate health warning. Battery-monitored devices health-check their battery sensors so a dead battery is never silent.
What this stack alerts on:
| Category | Examples | Priority |
|---|---|---|
| Safety | Water leaks (5 sensors), radon high, smoke/CO | notify.critical |
| Security | Door open + nobody home, garage left open, lock battery low | notify.critical / warn |
| Air quality | CO2 high in bedroom at night, PM2.5 high, pollen forecast | notify.warn / info |
| Battery | Personal devices < 15%, infrastructure devices < 20% | notify.info / warn |
| Appliance | Washer/dishwasher cycle complete (energy-monitor inferred), salt level low, filter needs change | notify.info |
| Pet care | Cat fed N times in 24h, water dispenser reservoir low | notify.damon |
| Package | Package detected at front step (LLM vision) | notify.damon |
| Vehicle | Car window open while raining, EV charge complete | notify.info |
Critical alerts have resolve actions — the garage door alert offers a “Close” button that calls cover.close_cover directly from the notification. The state machine survives HA restarts because state lives in input_select helpers, not in memory — a postmortem-driven fix after alert acknowledgments were getting lost on restart.
Calendar as configuration
Calendar events are sensor data — but they’re also the configuration surface for travel, presence, and wake-up logic. A bespoke google_directions integration owns the pipeline from event to leave time. It calls Google’s Routes API (the modern successor to the legacy Directions API), parses hashtags from the event summary and description, and computes leave times. It replaced the HACS Google Maps Travel Time integration, which only computes a travel duration — google_directions owns the entire pipeline from event text to departure time.
The hashtag DSL turns event text into travel configuration:
| Tag | Effect |
|---|---|
#drive / #transit / #walk | Set travel mode |
#bus / #subway / #train / #tram | TRANSIT mode + filter to a sub-mode |
#fewer-transfers / #less-walking | Routing preference |
#route U6 | Prefer a specific transit line |
#early 15 | Arrive 15 minutes early |
#buffer 5 | Add a 5-minute departure buffer |
#wakeup 06:30 | Override the computed wake-up time |
#pack laptop | Items to pack (read by the leave-time announcement) |
Per-person defaults live in input_select helpers — I default to #drive, the kids default to #transit — so events without tags still produce sensible routes. Tags are case-insensitive; explicit-mode tags are mutually exclusive (first one wins).
A three-layer blueprint pipeline turns this into per-person leave-time alerts: event discovery finds the next actionable event, the leave-time sensor calls get_leave_time and stores the result, and the notification blueprint fires at configurable intervals before departure. Notifications are dual-channel: a push notification on the person’s phone and a Sonos announcement in their current room — “Charlotte, you should leave for school in 10 minutes — the U6 toward Garching is at half past.”
Calendar-driven wake-up alarm
The same calendar surface drives wake-up. Each evening, when the master bedroom or kids’ room enters sleep mode, script.evaluate_wake_up_alarm runs. It queries tomorrow’s events from every family calendar. For each timed morning event, it computes alarm_time = event_start - 55 min - prep_buffer (35 minutes household-wide). The 55-minute departure estimate is a deliberate fallback — calling get_leave_time from the Routes API at sleep-evaluation time is the planned upgrade, but it introduces a dependency on a live API call at bedtime. For events with a #wakeup HH:MM tag, it uses the tagged time directly. It takes the earliest time across all qualifying events. If nothing is on the calendar, the alarm stays off — no weekday/weekend heuristic, no day-type fallback. One-off manual changes are overwritten the next evening, so there’s no stale state to manage.
The good night briefing announces the result: “I’ll wake you at half past six — Charlotte has school.”
The voice
The house speaks. A bespoke 400-line custom integration drives Gemini TTS to Sonos speakers in every room. It replaced ChimeTTS (a popular HACS integration) to solve three concrete problems:
| Problem | ChimeTTS | Custom announce solution |
|---|---|---|
| TTS timeout | 120s monolithic timeout for the entire request. Gemini generative TTS can take 10-60s for long messages. | Streaming SSE endpoint with a per-chunk timeout (default 30s). Data arrives every few hundred ms, so the timeout only fires if the stream truly stalls. |
| Voice quality | Re-encodes all audio through pydub (lossy MP3 conversion for Alexa compatibility). All speakers in this house are Sonos and support WAV natively. | Raw PCM from Gemini wrapped in a WAV header. No pydub, no ffmpeg, no lossy conversion. |
| Cache bloat | 256 cached files (34 MB), 91% older than 7 days. No automatic cleanup. | TTL-based cleanup on startup (default 7 days). Dynamic announcements auto-expire. Static phrases produce stable hashes and persist. |
The replacement streams audio from Gemini’s streamGenerateContent endpoint with responseModalities: ["AUDIO"]. Raw PCM arrives as base64-encoded SSE chunks, gets accumulated, and is wrapped in a WAV header (24kHz, 16-bit, mono). Sonos plays WAV natively via the AudioClip API, which ducks current audio, plays the announcement at a specified volume, then auto-restores playback. Multi-speaker playback is parallel via asyncio.gather — timing offset between speakers is typically 50-300ms.
The integration opportunistically waits for natural gaps between music tracks before playing. If a track boundary is imminent (within a configurable max_wait), it sleeps until the gap rather than ducking mid-song. A deterministic-hash audio cache (SHA256(message|voice|model)) with TTL-based cleanup ensures dynamic announcements auto-expire while static phrases persist. A pre-warming service (announce.generate) lets the wake-up automation generate audio at 5 AM so the actual announcement at wake time is instant.
Routing is person-aware: pass a person.* entity and script.announce resolves their current room via Bermuda BLE, then maps to the right Sonos speaker. Anyone away or asleep is skipped. DND filtering cross-references per-room and per-floor Do Not Disturb state before playing.
Morning briefings
Two briefing sensors fire daily — adults at 05:00, kids at 05:02 (offset to avoid LLM rate-limit collision). The HA side is small: the wake-up automation calls announce.generate on each briefing as soon as the sensor updates, pre-warming the TTS cache so the spoken briefing at wake time is instant. The content pipeline that fills those sensors — collectors, Haiku filter, Sonnet scorer, topic clustering — runs outside HA and is covered in Claude Code Workflow.
The wall panel
A tablet mounted in the hallway runs a dedicated dashboard in kiosk mode. It shows a greeting, weather, public transit departures, family presence status, and the car’s state. A lovelace-wallpanel screensaver cycles through family photos synced from Google Photos. The screensaver dismisses automatically for priority events like doorbell rings or security camera detections. Camera feeds are embedded as direct RTSP streams for sub-second latency.
Observability
A VictoriaLogs + Grafana stack on the Debian VM provides dashboards and ad-hoc log queries. An air-quality dashboard panels CO2, radon, PM2.5, and VOC over time per room. A Claude Code observability dashboard tracks agent activity and token spend. HA, container, and UniFi CEF logs are searchable via LogsQL for root-cause investigations. vmalert + Alertmanager fires log-based alerts to Discord.
Design philosophy
Blueprints over copy-paste. Any pattern that appears in more than two rooms becomes a blueprint. The blueprint handles the edge cases once; instances just supply parameters.
Bespoke at the edges, open source at the core. Home Assistant itself, the Sonos drivers, the Gemini SDK, Bermuda BLE, Alarmo, ZFS, Proxmox, Tailscale, Caddy, Syncthing, VictoriaLogs, Grafana — these stay firmly open source. When AI can produce a bespoke integration in a weekend, the integration glue at the edges becomes disposable. The announce integration, the google_directions integration, the presence_light blueprint, the alert state machine — all purpose-built for exactly this house.
The deliberative plane — the agents that write these packages, the SDLC pipeline that reviews them, the Todoist control plane that schedules the work — is Claude Code Workflow.