Yes, it’s not much different than a bowl of water being heated from one side and cooled by a heat sink with a fan on it on the other. It takes time to move the temperature of that water around with some fixed heat source/sink.
That's not quite the right explanation. With the pump in the loop, the heat transfer speed is actually quite high, and ideally the temperature difference between "after CPU" and "after radiator" is small. Without the pump (ie. "bowl"), the temp difference would be quite big until convection kicks in.
However, cooling on any radiator is a function of the temperature difference between air and metal. The heat from the CPU will slowly heat up the water and radiator, which has actually a large heat capacity. That takes times, so the temperatures go up slower. When the heater stops, it will take the radiator also comparatively long to get that heat out of the water again.
A normal air cooler has much smaller heat capacity, so will react much faster, in both directions. These faster cycles are certainly less good for the CPU than the slower cycles of a water loop.
