This is cool. I've previously wondered if this sort of system could be a net gain at all, but it sounds like it is. The last time I contemplated compressed in-RAM swap was when I was working working with a memory-starved game console, where swapping is normally unheard of. I never got to try it due to lack of resources.
The other idea that immediately springs to mind after reading this: what about compressing disk swap? Clearly the de-/compression is fast enough (~7-12µs), and the disk latency is the dominating factor. You might not save much, but I'd be surprised if it was worse.
With a swap partition located on a 10000 RPM disk, average time required for swap read and write requests was found to be 168ms and 355ms, respectively.
I find this rather interesting, as it's orders of magnitude above the theoretical time. Presumably this comes from I/O scheduling?
I'm aware that seek time is a serious problem, but I wonder if you could reduce the number of seek operations by reducing the number of disk I/O operations, full stop. Instead of each page corresponding to one 4kB disk block, you could stuff 2-3 in there and only need to seek and write once instead of 3 times.
If you could workout which 2-3 pages to put in there you could also just put them adjacently on the disk (assuming you weren't very low on swap space). You could do this the adjacent memory pages as the process sees it, however processes tend to assume they can jump all around memory without penalty.
Related, random idea that's just popped into my head: what about using compression on the disk cache in memory? This is sort of the opposite, when you've got more than enough RAM. (I have 8GB in my desktop, I rarely use more than 6GB for apps) The rest of the memory gets used as a cache for files, this could be much more efficient if compressed.
The other idea that immediately springs to mind after reading this: what about compressing disk swap? Clearly the de-/compression is fast enough (~7-12µs), and the disk latency is the dominating factor. You might not save much, but I'd be surprised if it was worse.