> Practically, it is foolishness, for what you save in money you lose in time and sanity.
Kubernetes gets a lot of side eyes in the self-hosted community. That's all of self hosting though. So why not go all in?
I've got 3 dell r720XDs running nixos with k3s in multi master mode. It runs rook/ceph for storage, and I've got like 12 hard drives in various sizes. My favorite party trick is yoinking a random hard drive out of the cluster while streaming videos. Does not care. Plug it back in and it's like nothing happened. I've still got tons of room and I keep finding useful things to host on it.
Plenty of people use k8s or k3s for self hosting. But for most, the added complexity doesn't buy enough for the trade-off to be worth it. Keep in mind most people have a single node, so docker does everything they need.
Personally, even with a 4 node setup (of tiny desktops; the hardware you have would easily cost me $200/mo in power bills), I use docker swarm. Old and unloved, but does everything I need for multi node deployment and orchestration with only a sliver more complexity than vanilla docker.
Yeah don't ask me about my power bill, It's definitely in the vanity realm. I have cheap power where I live so it's not anywhere near $200. Still too high though. One day I'll get solar to offset it.
I just use NixOS as a VM and run services as containers directly. Self-plug: I wrote a tool that makes it easy to run Docker Compose projects on NixOS [1].
This way, I get the advantages of NixOS config, while also being able to run arbitrary applications that might not be available on nixpkgs.
As far as storage goes, I just use ZFS on the hypervisor (Proxmox) and expose that over NFS locally.
Kubernetes gets a lot of side eyes in the self-hosted community. That's all of self hosting though. So why not go all in?
I've got 3 dell r720XDs running nixos with k3s in multi master mode. It runs rook/ceph for storage, and I've got like 12 hard drives in various sizes. My favorite party trick is yoinking a random hard drive out of the cluster while streaming videos. Does not care. Plug it back in and it's like nothing happened. I've still got tons of room and I keep finding useful things to host on it.