The protocol ensures that there is on average a block produced every 10 minutes.
As hash rates increase, and blocks are found slightly faster, the difficulty is adjusted upwards to ensure that the 1 block per 10 minutes is maintained.
We've seen difficulty drop in the past, it doesn't necessarily rise forever. It only makes sense to increase when it's still profitable to mine at the current difficulty.
If the difficulty rises to a certain level and the price falls, and it becomes unprofitable to mine for some miners, they switch off their rigs and the difficulty adjusts downwards after a period of time to compensate.
Over the years we have seen the price rise and hash optimizations made, which have both driven the difficulty upwards.
It's not strictly true in the sense that if the price were to fall over long enough periods of time, you would expect the hash rate to eventually fall, too.
But that's not plausible in the scenario where the world's financial system eventually runs on a proof of work cryptocurrency.
Since all miners compete over the same finite profits, each miner individually has an incentive to increase their hash power and therefore power consumption.
Even if the price was on average constant, the game theory would predict a competition over finding the cheapest way to burn the maximum amount of power.
Empirically, there were some transient drops in hash rate for both bitcoin and ethereum, on top of a constant massive run up.
Just because too much of a thing is bad doesn't mean the thing is bad in lower quantities.
Drinking ten liters of water in an hour is bad. That doesn't mean you shouldn't drink any water.