I really would have to disagree with the statement one should never use Azure Files over Azure disks.
1. Most Azure VM types have very stringent limits on attached disks; a K8s worker can easily blow past this limit.
2. You have tremendous complexity to deal with: pick Azure managed disks vs unmanaged disks on storage accounts (you can’t mix them on the same cluster). You have to understand the trade of of standard vs premium storage and how they bill (premium rounds up and charges by capacity, not consumption). And you need the right VM types for premium.
3. Managed disks each create a resource object in your resource group. A resource group last I checked had hard limits on the number of resources (like 4000?). Each VM is minimum 3 to 4 resources (with a NIC, image, and disk)... at scale this gets difficult.
4. Azure disks require significant time to create.
, mount and remount. A StatefulSet pod failure will sometimes take 3-5 minutes for it’s PV to move to a different worker. And worse when your Azure region has allocation problems. Azure files are near instantaneous unmoubt/remount.
5. Azure disks are block storage and thus only ReadWriteOnce. Azure files are RWM.
So, sure, if you’re running a cluster database with dedicated per node PVs and limited expected redeployments... use Azure disks. If you need a PV for any other reason... especially for application tiers that churn frequently.. use Azure files.
This is all true, I sort of forget that I still have a sort of Azure Stockholm syndrome. I'd say it's good feedback but it's nothing Azure doesn't know about.
Maybe Azure Files performance has improved to the point where it's more usable for storage scenarios. I suppose it probably comes down to the use case and application behavior.
It would be good if Azure had someone testing out these scenarios and interfacing with the larger k8s community, maybe through the SIG, for these sorts of musings and questions.
1. Most Azure VM types have very stringent limits on attached disks; a K8s worker can easily blow past this limit.
2. You have tremendous complexity to deal with: pick Azure managed disks vs unmanaged disks on storage accounts (you can’t mix them on the same cluster). You have to understand the trade of of standard vs premium storage and how they bill (premium rounds up and charges by capacity, not consumption). And you need the right VM types for premium.
3. Managed disks each create a resource object in your resource group. A resource group last I checked had hard limits on the number of resources (like 4000?). Each VM is minimum 3 to 4 resources (with a NIC, image, and disk)... at scale this gets difficult.
4. Azure disks require significant time to create. , mount and remount. A StatefulSet pod failure will sometimes take 3-5 minutes for it’s PV to move to a different worker. And worse when your Azure region has allocation problems. Azure files are near instantaneous unmoubt/remount.
5. Azure disks are block storage and thus only ReadWriteOnce. Azure files are RWM.
So, sure, if you’re running a cluster database with dedicated per node PVs and limited expected redeployments... use Azure disks. If you need a PV for any other reason... especially for application tiers that churn frequently.. use Azure files.