jerry@homelab:~/homelab$ cat homelab.md
The homelab
Everything on this site runs on, or because of, a small homelab: a few machines at home and two free cloud VMs, with every service in a container and everything managed as code. This page is the overview; the projects are the things I built on top of it.
The machines
They're named after places in a post-apocalyptic world.
| Name | What | Role |
|---|---|---|
| Wasteland | Intel NUC12, Ubuntu | The main server: reverse proxy, single sign-on and most apps |
| Badlands | Intel NUC10, 16 GB, Ubuntu Server | The second server: finance, travel, calendar, mail client, automation |
| Beacon | Oracle Cloud free-tier VM, 1 GB RAM | This website and its visitor stats |
| Watchtower | Oracle Cloud free-tier VM | Watches the home network from outside |
| Checkpoint | Raspberry Pi 3 | The backup DNS server |
| Junction | Netgear managed switch | Connects the home servers |
| Sanctuary | Laptop | |
| Outpost | Phone |
What runs on them, roughly: photos (Immich, with hardware video transcoding and machine learning on the NUC's GPU), documents (Paperless-ngx), passwords (Vaultwarden), git and CI (Forgejo), notes, RSS, calendar and tasks, a webmail client, budgeting, subscriptions, travel logging, workflow automation (n8n), a photo frame on the living-room TV, monitoring, and three Minecraft servers. Plus the things on this site: my own start page, Minecraft tool and morning briefing.
Nothing exposed
The rule everything else follows: nothing at home is reachable from the internet.
- No router port-forwards, not even for Minecraft. The game servers are LAN-only.
- The reverse proxy binds only to the LAN address and the Tailscale address, never to all interfaces. Away from home, my devices reach everything over Tailscale.
- Real HTTPS anyway. Every internal subdomain gets a real Let's Encrypt certificate through a DNS challenge at my registrar, so nothing needs a self-signed certificate or a public port.
- The only public service is this website, and it doesn't run at home. It has its own cloud VM and its own proxy, and its visitors never touch the home network.
One entry point
Caddy on Wasteland is the single entry point for every app, including the apps that run on Badlands: for those, Caddy forwards to Badlands over Tailscale, and DNS for every subdomain points at Wasteland. I looked at a shared Docker network across both hosts, but that means Docker Swarm, a whole extra layer, for a problem a reverse proxy already solves.
Inside a host, containers talk to each other by container name over a shared Docker network, never through the host's own address. That rule came from a cryptic tls: internal error: a sync job aimed at the host's IP actually hit Caddy, which had no idea which site a bare IP was asking for. Databases sit on private per-stack networks that nothing else can reach.
DNS
Pi-hole does ad blocking and split-horizon DNS: at home, every subdomain resolves straight to the server. A second Pi-hole on the Raspberry Pi is a real failover, not an idle copy: the router hands it out as secondary DNS, and Tailscale lists both. A sync job copies blocklists, settings and local DNS records from the primary every hour.
I decided against Tailscale subnet routing, which would give every device on the tailnet access to the whole home network. The curated set of subdomains already covers everything I need away from home.
Single sign-on
Most apps sit behind Authelia, in one of two ways, with one rule to choose:
- Real OIDC login for every app that supports it: Immich, Paperless, Forgejo, Miniflux, the budgeting and travel apps, and more. Each shows a "Sign in with Authelia" button.
- A forward-auth gate only for apps with no login of their own, like the Docker dashboard and Pi-hole: nothing reaches them until Authelia says yes.
- Never both. A gate in front of an app that also logs in through the same Authelia is the same check twice, not extra security.
- No silent accounts. An Authelia login never creates a new account in an app; accounts are linked deliberately.
A few things stay outside it on purpose: the password manager (its own strong login; a second one in front is just friction), the calendar server and the TV photo frame (their clients can't do a browser login), and a handful of harmless tools.
The lesson that changed how machines talk to each other: a headless client hitting a gated domain fails in ways that don't look like auth. A backup script's status push "succeeded" because curl -f treats Authelia's redirect to the login page as success. A remote agent reported websocket: bad handshake. Since then, machine-to-machine traffic goes to container names or Tailscale addresses directly wherever it can, past the login gate. The few clients that must use a public address get an explicit bypass rule, and all of those live in one place: Authelia's own config.
Monitoring
Two Gatus instances, deliberately not identical, because they answer two different questions:
- At home: "is this app alive?" It checks every app directly by container name or Tailscale address, past the proxy and the login gate, so a check reflects the app itself. It also receives the backups' status pushes (below), and it asks mc-tool about the Minecraft servers, so a server asleep on purpose isn't an outage.
- From the outside VM: "can the whole place be reached?" A monitor on the network it watches can't notice that network going dark. This one checks the login portal as a canary, this public website, and the domain's expiry. Its alerts also go by email, because the Signal bridge runs at home, and if home is down, so is Signal.
Beszel watches the hosts themselves: CPU, memory, disks and drive health, with alerts to Signal. Every host's containers show up in one Docker dashboard through small agents that connect out to it, so no host needs an open port for that.
Backups
- Local and offsite: a nightly local restic repository, plus five restic repositories on Backblaze B2, one per job, each with a key that can only reach its own bucket.
- Consistent databases: the backup stops the database containers first, and a cleanup handler starts them again on every exit, including a failed backup.
- A dead man's switch: every backup reports its result to Gatus on every exit, success or failure. If a report doesn't arrive in time, Gatus alerts, so a cron job that silently stopped running gets noticed too.
- Staggered: the jobs are spread over the night, so they don't fight over disks and bandwidth.
- More than 3-2-1: four copies in three physical places.
- Tested: I've restored the main server's data, the photo library and the second server's data for real. A backup that's never been restored is a hope.
The rules that keep it simple
- Everything is code. Every stack has its own git repository and deploys on a push (how). Secrets never go in a repository: they live in
.envfiles on the host or under/etc. - Hosts are addressed by IP, not by name, in every config and script. When I renamed the cloud VMs and the Pi to the names above, nothing broke. The one surprise was restic: it groups snapshots by hostname, so the old snapshots would never have been cleaned up without telling it to group by path instead.
- Write down what went wrong. Every gotcha gets written up with its root cause and fix, because the next time it happens, it's usually me who's forgotten.
jerry@homelab:~/homelab$ โ
cd ..