The solution is to additionally stuff a bigass electrolytic capacitor on the rail, either tantalum or aluminum. The ESR of the electrolytic will damp out the naughty high-Q tendencies of the ceramics and everything will work out wonderfully in typical cases. (Of course there are pathological cases out there. There are always pathological cases. And if there ever stop being pathological cases, I'll be out of a job!)
At this point—thank you!—a Zachtronics game was born in my head. I’d like to play it!
(Maybe it’s a good secret level in that Zachtronics game about nondeterministic infinitesimals portrayed as getting things done in a corporate environment;… what was the name of that one again?)
