I have no doubt that Google’s search team is optimizing for the best results. The problem is their ads team is optimizing for revenue. You can’t optimize for two things at the same time without compromising (the optimum is the Pareto frontier).
In my opinion, the issue is that the user's definition of "best results" and Google's definition do not align, including the Search team. Google's incentives are very different than user's needs.
I disagree, personally. I think the engineers there are human and they can see that the results aren’t good. Maybe some of them are in this thread. If I had to guess, I think there are a lot of sub-teams contributing scores to the overall ranking and optimizing for sub-metrics, and it’s too big of a beast to fix. In other words, a failure of leadership.
Of course. But this will be true for OpenAI and all other AI players as well. During the honeymoon phase user needs matter, that’s how they get you. Comparing a company in the free-money phase with one in the value extraction phase is apples to oranges.