If I specify a logical size, and that's larger than the physical backing size, then what happens if someone attempts to write unique high-entropy data to fill their logical space?
VDO will return ENOSPC when it gets a write and has nowhere to store it. The filesystem or other writer is responsible for passing ENOSPC up to the user, just like if dm-thin were to run out of space.
OK, that sounds like a convincing reason not to use this feature - it really looks like it covers wrong abstraction. Consider reasons why you could get ENOSPC before:
1. Ran out of disk space,
2. Ran out of inodes,
3. Media reported ENOSPC and it gets propagated,
4. (something else I don't know about?)
So now, #3 gets more common. Is it actually something that FS devs think about? "What if I get this weird error message?" Or is it something that could trigger a some destructive edge case in the filesystem and lead to serious data loss? You mention dm-thin doing the same, so I guess that at least the popular filesystems should handle it well.
Anyway, when you think about it, how do you debug #3? You checked file size, you checked number of inodes, other than that the FS gives you no feedback. There's no central API to signalling this kind of problems and suggested solutions and I would say that this makes systems much less flexible. You could of course run vdostats (and whatever dm-thin uses to report its resources), but just imagine the amount of delicate code you would need to automatically solve this kind of issues. It's insane, it really looks like crappy engineering to me.
#3 is certainly not a historically common error from a disk, but thin provisioning has been around for a while -- dm-thin was introduced in Linux 3.2 in 2012, and filesystems in my opinion worked extensively to handle ENOSPC correctly since then. Consider that storage returning ENOSPC at a particular write and all thereafter is roughly equivalent to storage stopping writing suddenly, and storage stopping writing is equivalent to a system crash and reboot. Filesystems do work hard to recover without loss of unflushed data from a crash, assuming the storage was properly processing flushes, and this case should be very similar.
Filesystems (and VDO) both log extensively to the kernel log in an out of space situation, so inspection via dmesg or journalctl hopefully leads swiftly to identification of the problem. The 'dmeventd' daemon provides automatic monitoring of various devices (thin, snapshot, and raid) and emits warnings when the thin-provisioned devices it is aware of are low on space; there's a bug filed against VDO to emit similar warnings [1]. Careful monitoring is definitely important with thin provisioned devices, though.
inodes are a similarly weird reason to be “out of space”. the number of new-to-linux users that i’ve asked “have you checked inodes?” is huge. once you know that, you know that there are more things than just bits on the drive that effect available “space”
And deduplication, yes.
>Can I grow the logical size later?
Yes; the man page for the program discusses the growLogical subcommand briefly: https://github.com/dm-vdo/vdo/blob/6.1.1.24/vdo-manager/man/...
>I can also see that we're specifying a "logical size" that is exposed to the user. How much space is really used and how is it allocated?
There's a vdostats command mentioned in the article that exposes how much actual space is available and used.
>Apart from this - what is the status of the patch, is it redhat-only or is it on its way to the upstream?
It has not yet been submitted as a kernel patch.