My main goal is to replace cloud services so I can be Google-free. I've also got LineageOS + MicroG on my phone. This is all running in docker containers on NixOS (other than OPNSense), with automated restic backups to a NAS as well as Backblaze. One of my goals is to be able to deploy all this again from scratch with minimal effort, and I think I succeeded, though haven't had to test it yet.
Nextcloud - for caldav and carddav calendar, contacts, and tasks
Xbrowsersync - sync bookmarks across device
Synchthing - backup data from my phone. I use Neo Backup to take a snapshot of all apps, so the phone should theoretically be restorable from scratch.
Jellyfin - Spotify replacement. The Finamp app is fantastic.
Home Assistant - automate my media center, as well as control outdoor lights and door locks, and check if any doors or windows are open or unlocked when I'm away.
OPNSense on a protectli box - amazing open source gateway software that does everything.
AdGuard Home (on OPNSense) - DNS based ad blocking
Wireguard (on OPNSense) - allows me to have an always on partial tunnel VPN on my phone and laptops that allows access to home services while remote, and also allows me to use my Ad Guard DNS.
HAProxy + LetsEncrypt (on OPNSense) - setup to provide subdomains for each of the services at home. Only a couple are public (contacts and calendar), but the rest become available when the VPN is on.
Smokeping - use it to collect data to rub into Spectrums face when they go down.
How much time do you spend maintaining this stuff every month? Like upgrading versions, etc.? Do you subscribe to any security channels? Do you care what language it's built in, try to keep it all on a few languages to minimize maintenance burden, etc.? I've been considering self-hosting, but put off thus far by the above concerns.
Initially I spent a lot of time as I used it as an opportunity to learn Nix/NixOS. I used Nix intentionally as it's a rolling release and also it's declarative and intended for reproducible deployments, so I don't need to deal with an OS like Ubuntu that slowly gets crufty and out of date and needs a clean-up or upgrade or complete re-install. And if I do need to re-install, it should be mostly a one-liner.
I also run all services in docker and my network uses VLANs behind an OPNSense firewall. I use Wireguard as a pinch point into my network to access most services. So I'm not too worried about the security aspect.
Upgrading on Nix is pretty easy - just bump your lock file and it will get the latest packages, assuming you are on the unstable channel. But unstable does break on occasion. You an also use the latest stable release of Nix and selectively choose unstable packages, which is probably the way to go. I rarely need to fix anything - it's been pretty reliable - I feel like with each patch I make to my config (which is all checked into git), the system gets more reliable and reproducible - 2 steps forward, no steps back. It only starts eating time when I want to add or upgrade some element to the system, but I always make sure to never do any action that isn't captured in Nix config and backed up, so that I don't have to come back and figure out what exactly I did or how something works again. It's been fine. Nix has a pretty steep learning curve, but considering its power, I think it's absolutely worth it.
* Use Agenix to store secrets out in the open (encrypted with your SSH key)
* Use home-manager
* Keep config modular so you can selectively include it in various hosts and domains (e.g. desktop vs server).
Think of it as building software when you figure out how to organize your config. The bulletpoints above can all be a bit difficult to figure out so don't feel bad if you don't get them installed right away. Try to start with a machine that isn't your daily driver and blocking your workflow.
Any thoughts on Guix as compared with Nix? I've tinkered a bit with Nix, including running NixOS for a bit on my laptop, but quite like the lispy languages in comparison.
I found out about guix after already investing in nix. Looks great, and better in many ways, but I'm more familiar with Haskell syntax than lisp, and my understanding is that the nix ecosystem is far more developed.
Good to know, thanks! It does seem to have more packages and such for sure, and more traction among Haskell crowd I think from the bit I've seen.
Either way, I am more inclined currently to use packages on top of a base Linux distro, so suspect I can mess around more without committing to either for now.
How do you handle partition management? I've got all the mount points set up in my configs but the initial creation of partitions was still a manual process that I'd need to go back to my notes for if I ever have to redeploy things.
I wrote a script that I have in GitHub that I curl down to a new system I need to prep. It takes four or five params, e.g. UEFI vs legacy, SSD vs disk, etc, then does all the prep programmatically. This really should be part of the nix ecosystem though.
Started by getting it working on my new work laptop (Thinkpad), then putting it on my personal laptop (Dell XPS) to figure out how to split config into multiple hosts, then finally onto an HP proliant server I've got hosted at home.
100%. I did already have another laptop that I used until this one became stable. Now I feel it's more stable and useful than my previous one though. It actually helps rather than hinders when it comes to avoiding messups and catastrophic mistakes. Each time you deploy, it creates a snapshot, so you can always roll back if you mess something up. And as you update your config, your system gets progressively more stable and to your liking. And it's forever, since it's just text files stored in a repo. I feel like it achieves the same thing as Chromebooks do, except without the involvement of a centralized corp. My laptop is my data rather than the hardware itself. You do need to backup anything stateful though, but I don't have much stateful data on my machine other than some code in my home folder which is already backed up in git.
You can run this stuff using docker & then just have a cron script to `docker-compose pull && docker-compose up -d`. I run this once a week and it keeps all my software up-to-date.
Not OP but I was running a number of services with docker.
This was a major pain. Docker is fine for local development, but unless you have a full time department dedicated to babysit it, it’s not scalable for self hosting. Observing services becomes complicated and requires a whole tool chain, ditto keeping on top of new versions, etc.
What I did instead is install proxmox on my server, and run the former server (that was running all the docker stuff) as a virtual machine within proxmox.
I was then able to gradually move everything away from docker and into VMs or LXC containers (which are basically lightweight VMs, but you can use all existing tooling and treat it like a VM). I’m using ansible roles and playbooks, I can ssh into the containers, and unattended upgrades for updating, and tarsnap or borg for backups.
It’s much saner this way ad I don’t end up running versions that are several years old because of the complexity/inertia of docker.
Not OP, but I'm self-hosting for similar reasons. I don't spend very much time on maintenance - if a service is troublesome, it's not worth my time and gets the cut. I don't subscribe to any security newsletters or have any automated new version monitoring in place (yet), but I'm planning to do that at some point so that the home lab can run itself with minimal intervention beyond changing a few version numbers in docker-compose files and rerunning my "redeploy everything" script.
As for languages, it's a very marginal factor. If I have a large number of possible services in a given space, I usually gravitate towards Go, because most Go applications are closer to 12-Factor compliance and generally stability than, say, most Python applications. If I have no options or the best service in a given space is written in COBOL or something equally bad, I'll still take that. I'm quite pragmatic.
IMO, if you want to get into this, keep a few things in mind:
- Docker (Compose) will save you a lot of work
- if you want to expose stuff to the internet, you'll need to stay reasonably current with updates
- have a good system for backing up your data. Experimenting is far easier if you can back up what you've got, and start over from scratch to try a different approach. Doing this "live" is riskier.
Funny, I used smokeping to run regular speedtests on top of pings to establish the cable connection in my neighbourhood was oversubscribed (daily slowdown to a crawl during work hours in WFH mandate, order of magnitude increased ping during the day time vs e.g 3 a.m.). Changed connection/provider and would consistently get max speeds and more consistent ping.
Who was your provider, and who is your new provider?
I was having both bad connectivity to Spectrum as well as buffer bloat, which I think was upstream. Had to get Spectrum in three times, after which the spent several days up on the pole and in the field doing major work, and the problem resolved.
This was in Germany, using Vodafone (Kabel Deutschland), moved to "supervectoring" DSL and life was better (it's not fiber, but it's second best) - wrote it up at the time here: https://arahayrabedian.github.io/cable-to-dsl/
My neighbors still have problems with the cable connection.
Hi! I have been wondering about whether investing into home assistant would be worth it to control my media center as well. Do you happen to have handy links to any resources you found helpful?
Not OP but it makes the whole setup easier to use.
For instance, when I turn on the PS4, HA turns on the receiver and the TV, switches to the correct inout, adjusts the lights in the room.
When I turn off the ps4, it switches off the tv and the avr, unless I switched the receiver to the music or media player input (in which case it turns the tv off or not)
It also lets me use these cheapo ikea zigbee volume buttons to adjust the sound on the receiver, pause and skip songs or video (via libreelec). And the “light switch” aqara button to turn off all lights at bedtime from my bed and keep the music running (if I double tap) or switch it off otherwise.
The above wouldn’t be possible otherwise: ikea’s volume switch only works with these ikea/Sonos lamp things otherwise.
Not OP, but various automations that fire off commands based on whatever your TV, receiver, Apple TV box, etc are doing are how I find it most useful.
Example: I have some cheap Govee LED strip lights behind my TV for ambient lighting. HomeAssistant can detect when my Apple TV (or Samsung smart TV) is on and automatically turn on the lights for me. I don't have to reach around the back of the TV to try and find the little button to turn the lights on (or remember to turn them off).
I can also control both my TV and Apple TV through HomeAssistant. It's not exactly the most polished/straightforward, but you could definitely string together some automations - something like a "movie night" button that dims the lights, turns on the TV, switches to the appropriate input, and cues up a file. For me that's more hassle than it's worth.
> I've got an msi desktop gaming PC, an LG CX OLED TV, and a Yamaha RX-A2A receiver and they never played well together. The kids always had a hard time getting them all on at once and set to the right inputs and launching steam.
> So I created a Home Assistant automation that does all that, bought a Zwave button that sits on the coffee table, and now they just turn it all on with one button like it's a video game console.
I also plan to add "scenes" where I can just tap the button and the lights dim, and the media center gets put into movie mode, as well as a "music" scene for when I have parties, which would join the two zones my receiver supports an then
start playing a playlist from spotify.
Love this. My TV setup is super straightforward these days, but I had a nice home theater setup in my previous house and used a rather disappointing Logitech Harmony remote.
If you want to get creative, you could create a custom dashboard and put an old iPod touch/Android device in kiosk mode and use it as a remote touch panel control for your home theater (or anything else in HomeAssistant).
I have two Lenovo M8 tablets ($100/each) that I'm using as home control panels - super convenient and rock solid. https://imgur.com/a/f0aNTRq
Yeah they're solid - come with a little dock so it looks like a high-end automation system panel. You can configure the power settings to hold the battery charge around 50% to prevent any issues with the battery swelling.
If you go this route, definitely buy the Android app FullyKiosk. It will let you lock the tablet to the HomeAssistant dashboard, automatically recover if something crashes, etc. I have it set up to use the built-in camera & motion sensor to automatically turn on the display if someone walks up to it or touches the tablet, and automatically turns the screen off after a few minutes of no motion.
Would defianately test it, i tried my backupstack and there was an issue i couldnt have reverted, so it might look fine on the surface but actually doing it is the only way to make sure!
Hey, super interested in your OPNSense install. I also have a ProtectCLI box. However it currently runs PFSense.
Whenever I try to install OPNSense it fails to load once installed. Maybe there are some initial configuration steps that I am missing? Last time I tried this, about a month ago, internal DHCP addresses were not getting assigned to clients. Troubleshooted for an hour, no results. So back to PFSense I went.
Do you have a guide for installing and configuring the basics? Or something you would recommend?
Hmm, I installed it a long while ago, and don't recall what guide I used. One of the benefits though of buying a Protectli box rather than the original Qotom version is the is support. Protectli should be able to get you up and running - check their website and get in touch with them.
Could you link to Pintry? I couldn't find it from a quick Google search.
I finally found a use case for Pinterest after creating an account years ago and their landing page refreshed and acted weird so much in Firefox that they decided it was a phishing attempt and locked my account for some arbitrary amount of time. I'd rather not even start using it if there's a viable alternative
Nextcloud - for caldav and carddav calendar, contacts, and tasks
Xbrowsersync - sync bookmarks across device
Synchthing - backup data from my phone. I use Neo Backup to take a snapshot of all apps, so the phone should theoretically be restorable from scratch.
Jellyfin - Spotify replacement. The Finamp app is fantastic.
Home Assistant - automate my media center, as well as control outdoor lights and door locks, and check if any doors or windows are open or unlocked when I'm away.
OPNSense on a protectli box - amazing open source gateway software that does everything.
AdGuard Home (on OPNSense) - DNS based ad blocking
Wireguard (on OPNSense) - allows me to have an always on partial tunnel VPN on my phone and laptops that allows access to home services while remote, and also allows me to use my Ad Guard DNS.
HAProxy + LetsEncrypt (on OPNSense) - setup to provide subdomains for each of the services at home. Only a couple are public (contacts and calendar), but the rest become available when the VPN is on.
Smokeping - use it to collect data to rub into Spectrums face when they go down.
Pintry - Pinterest clone