The point is, without a reasonably clean starting point, you'll never get to apply those 20-25x optimizations in code bases beyond a certain size, because they've become a wasteland of mediocre micro-optimizations before you even get started.

So the sensible thing to do is to identify a component of manageable size, fence it off with suitable abstraction boundaries, and then apply optimizations WITHIN the confines of that component only.