This is a great approach which resonates with me a lot. It's really frustrating that there is no simple way to run a multi-host Docker/Podman (Docker Swarm is abandonware since 2019 unfortunately).
However, in my opinion K8s has the worst API and UX possible. I find Docker Compose spec much more user friendly. So I'm experimenting with a multi-host docker-compose at the moment: https://github.com/psviderski/uncloud
Wouldn’t argue with you abut the k8s ux. Since it has all the ground concepts ( service, cronjob etc ) it required less effort than rolling yet another syntax.
Thanks! I'm more than happy to catch up to discuss the challenges. Feel free to reach out to me on twitter @psviderski or email 'me at psviderski.name'
The only problem is Docker Swarm is essentially abandonware after Docker was acquired by Mirantis in 2019. Core features still work but there is a ton of open issues and PRs which are ignored. It's fine if it works but no one cares if you found a bug or have ideas on how to improve something, even worse if you want to contribute.
Yep it's unfortunate, "it works for me" until it doesn't.
OTOH it's not a moving target. Docker historically has been quite infamous for that, we were talking about half-lives for features, as if they were unstable isotopes. It took initiatives like OCI to get things to settle.
K8s tries to solve the most complex problems, at the expense of leaving simple things stranded. If we had something like OCI for clustering, it would most likely take the same shape.
Generally speaking, redundancy/availability could also be achieved through replication rather than automatic rescheduling, where you deploy multiple replicas of the service across multiple machines. If one of them dies, the other one still continues service traffic. Like in good old days when we didn't have k8s and dynamic infra.
This trades off some automation for simplicity. Although, this approach may requires manual intervention when a machine fails permanently.
Working on Uncloud [1] — open source tool for self-hosting containerised apps across multiple machines. You can combine cloud VMs and on-premises to create hybrid compute environments, e.g. for cost optimisation or data privacy.
While Kubernetes offers power at the cost of complexity, Uncloud focuses on simplicity for common deployment workflows.
Progress from this month:
- Enhanced Docker Compose support: You can now deploy your entire stack from standard Docker Compose files. This includes volumes, environment variables, resource limits, scaling and logging configuration.
- Volume management: Create and manage Docker volumes across your cluster with automatic scheduling based on volume location.
- Context management: CLI command to quickly switch between multiple clusters, e.g. homelab and production one
I'm particularly excited about the volume management system as it provides the cluster semantics to the good old Docker volumes. It uses a constraint-based scheduler that ensures services sharing volumes are properly co-located.
If you're seeking something between "just Docker" and full Kubernetes for deploying applications on your own infrastructure, I'd love to get your feedback on Uncloud.
I'm continuing to develop Uncloud [1] — an open source tool for deploying and managing containerised applications across multiple Docker hosts. Includes WireGuard overlay network, ingress with automatic HTTPS, and simple Docker-like CLI. Unlike traditional orchestrators such as Kubernetes, Uncloud has no state reconciliation or quorum needs which makes it easy to understand and use. If you ever wanted something simpler than Kubernetes for self-hosting your apps on cloud VMs or in your homelab, Uncloud might be for you.
Key updates from the past month:
- New demo screencast [2]: Deploy a highly available web app with automatic HTTPS across cloud VMs and on-premises in just a couple minutes
- Added initial Docker Compose support for service deployment. The same Compose can be used for developing locally and deploying to your cluster
- Completely revamped how service and container specifications are stored, enabling proper implementation of the service 'scale' command and selective container recreation
My goal for Uncloud is to create a more capable and modern replacement for Docker Swarm, which is no longer being actively developed.
I agree there is no one-size-fits-all for data replication. Currently Uncloud doesn't handle volume replication. Moreover it doesn't support regular Docker volumes yet, only mounting a host path. The reason is I didn't have time to give it proper thought on how to design volumes in a cluster context without getting into the full-blown PV support like in K8s.
I suspect that I will implement support for regular local Docker volumes such that each service container will use its own volume on the machine it runs on. Uncloud won't automatically replicate data between volumes as storage replication adds significant complexity and potential failure modes. Apps that need HA such as databases can handle their own replication. I'm getting inspiration from Fly for this: https://fly.io/docs/volumes/overview/. Maybe it would make sense to implement handy commands for cloning, moving, and backing up volumes between machines, not sure yet.
Corrosion (the embedded CRDT SQLite) is only used for Uncloud's internal state management, not for application data.
I see, so off the top of my head I’m thinking if uncloud could host some of my containers: nextcloud, plex, radarr, transmission, listmonk, linkding. And it seems its no better than my single node k8s, i.e they all have either sqlite or filesystems state, and no clear way for HA replication… mind you this isnt a problem of uncloud, moreso of the apps themselves
Yep agreed, but on the other hand do you really need HA for them when you're perhaps a single user? I wonder, what motivated you to migrate to single-node K8s from Compose for your self-hosted setup?
No I don’t really need HA, for most of my services high availability comes last. I moved to k8s for learning purposes, I stayed because I like it. When I decide to move some services to HA I have a much more powerful base with k8s than compose.
Working on Uncloud[1][2] — think Fly.io but self-hosted. A lightweight Docker clustering tool for running web apps on your own servers (from cloud VMs to bare metal) with no control plane to maintain. Perfect for teams who want cloud-like deployments without Kubernetes complexity. Early days but seeing promising results with eventually consistent state sync, zero-config mesh networking, and automatic HTTPS ingress.
That is super useful, especially for small stuff in home labs. I was thinking to go ProxMox + Gardener + Kubevirt route, but never have time to work on it.
Oh this is going to be a beast. It depends on what you're trying to achieve though. If you're doing this primarily to learn the kube landscape then it's a great way to play around with these tools in a homelab.
As far as I know, Coolify is great for single-server PaaS or multi-servers that work independently. Uncloud focuses on distributed multi-server HA deployments (which is what is usually preferred for production). It allows you to spread workload replicas across different providers and locations while maintaining unified management. It's closer to Kubernetes and Docker Swarm in terms of providing building blocks to build your PaaS of choice.
Unlike Coolify which requires a dedicated server to run the control plane Uncloud has no central component to manage. It's just a network of interconnected machines so there's no single point of failure and no central service that needs updates, backups, etc.
Honestly, I went with Docker simply because I'm much more familiar with it and its networking stack made integrating with WireGuard overlay straightforward. Podman support could come later but Docker's popularity and API stability made it the practical choice for the initial implementation.
reply