I Don't Maintain My Homelab

This is surprising to me and the exact opposite of what I want for a few reasons: 1. I don't like surprise breakages. I am not prepared to fix a service my family uses midday on a Tuesday when I am working since it auto updated. I'd like to specifically make sure I have dedicated time and plan if something is going to go wrong. 2. My family HATES when things change. I try to run LTS versions of things, but annoyingly, some software like nextcloud doesn't have LTS version. One of the things my family likes the most, is that the stuff I host isn't constantly changing like commercial products. Having google photos change or netflix have a new interface randomly is very, very frustrating for them. Since my homelab is completely internal, I avoid quickly doing updates (unless it is a critical security issue), and definitely avoid doing major version upgrades unless there is good value in it.

I've had "servers" or a "homelab" at home for de3cades. I stopped a while ago when I burned out. About 4 month ago, I bought a new motherboard and graphics card for my desktop and dropped the old ones into a $70 case I got from Best Buy and put Ubuntu on it. I think I spent 10x that on memory for my new desktop, but that's just a passing grumble. The new server now runs transcription and embeddings for me on the old GPU. That motherboard is still plenty fast, but pushing 8 years old now. That's the advantage of buying a nice board from the outset. The rest of the lab is a few ephemeral instances on Google, with dual A100s that spin up when I need to train things. I put Ubuntu on the old beast, and never touch it. If the power goes out, it automatically comes on and Docker launches all the services when it comes up. About the only thing that needs watching is the tiny SDR radio plugged into it, which I use for pure random numbers and talking to it with a hand held radio from the other house. Sometimes I have to unplug it and then plug it back in to get it back into service. No amount of finagling seems to fix it from software.

I also have a "homelab" with minimal maintenance requirements. I'd wager it works out to much less than 15 minutes a month over a year. The strategy is as follows: pin all services to known good versions, deny access from outside LAN, and don't touch it unless there's a new service release with new features I want. Not something I would do at work, but perfectly fine for home setting.

This is a fantastic article! I completely agree with the author's philosophy. Simple automation can reduce maintenance to nearly zero, and it's incredible how much can be achieved with just a few well-crafted scripts. I use a nearly identical alias for docker pull to keep my containers updated. To ensure everything stays running smoothly, I've built a lightweight watchdog (a mix of bash scripting and Uptime Kuma/Beszel) that monitors my services and containers and restarts them if they crash. This way, I rarely need to intervene manually. For critical services (DNS, VPN, git, web search, crawler and mail, etc.), I add an extra layer of redundancy by running them on multiple servers across different locations. If one server fails, the others seamlessly take over. I also use DNS round-robin as a simple but effective way to handle load balancing and failover; no HaProxy, K8, expensive IP Takeover (ARP Spoofing) or BGP Anycast and VRRP/CARP, Proxmox or fancy orchestration tools required. If a node goes down, another watchdog script temporarily removes it from DNS, and traffic shifts to the remaining servers. Most often the services are self-healing. The best part? My deployment and monitoring are fully self-scripted (no Terraform, Ansible or BundleWrap). Moving services to a new server is as easy as running some scripts over SSH. Everything sets itself up automatically. Currently I run my services on 2 Pi's, 2 stratum 1 servers (from centerclick), and 8 VPSs that cost me around $40/month. It's a great example of how a little automation and redundancy can go a long way in keeping things cheap and reliable without unnecessary complexity. I invest around 1-2h/month to maintain and (mainly) adjust my setup. Before I head multiple Proxmox instances and a backup server that cost me around $250/month, I was spending 1-2h/week just to keep everything running. The difference is night and day. Thanks for the inspiration; it's always refreshing to see others embracing simplicity!

I thought this was going towards the "I have an agent do it". glad it didn't :) What this skips though is the complexity of services like NextCloud (stuck in maintenance mode again?), Immich (needs a compose file edit?), MineCraft worlds (Dad! my client is on another version again!), (dmn) AlbyHub (needs re-login and closed its channel). But to be fair this is really getting quite minimal these days indeed. I didn't really realize it but I too have a mostly hand-off home-lab... Ok, then it's not really a lab anymore, its more "stable home-infra" ;)