Which container runtime are you using? As far as I know both Docker and containerd use overlay filesystems instead of loopback devices.
And how did you know that tweaking the sector size to equal the underlying filesystem's block size would prevent double caching? Where can one get this sort of knowledge?
The loopback devices came from a CSI which creates a backing file on the kubelet’s filesystem and mounts it into the container as a block device. We use containerd.
I knew that enabling direct io would most likely disable double caching because that is literally the point of enabling direct io on a loopback device. Initially I just tried enabling direct io on the loopback devices, but that failed with a cryptic “invalid argument” error. After some more research I found that direct IO needs the sector size to match the filesystems block size in some cases to work.
And how did you know that tweaking the sector size to equal the underlying filesystem's block size would prevent double caching? Where can one get this sort of knowledge?