>Good points, thanks. What makes you wish you didn't use Rook/Ceph/Longhorn?
It seems like my volumes are constantly falling into degraded and then rebuilding. Resizing volumes requires taking the workload that's attached down, and then it seems to take forever (15m+) for my clusters to figure out that the pod is gone and a new pod is trying to attach.
Really, it's a PITA and all of the providers' storage classes seem better than Longhorn. Ceph I had less experience with but very similar problems - long-gone pods held a lock on PVCs that had to be manually expunged, or wait for a very long timeout.
I've had similar issues with Mayastor (another in-cluster storage solution). It's under heavy development, so I've assumed the more mature options were better.
I'm working on v2 of my homelab cluster, and I'm going with plain old NFS to a file server with a ZFS pool. Yes, I will have a single node as a point of failure, but with how much pain I've had so far I think I'll be coming out ahead in terms of uptime.
It seems like my volumes are constantly falling into degraded and then rebuilding. Resizing volumes requires taking the workload that's attached down, and then it seems to take forever (15m+) for my clusters to figure out that the pod is gone and a new pod is trying to attach.
Really, it's a PITA and all of the providers' storage classes seem better than Longhorn. Ceph I had less experience with but very similar problems - long-gone pods held a lock on PVCs that had to be manually expunged, or wait for a very long timeout.