And you're simply wrong. Go find out the price of any random lot near you over a long time period, if you don't believe me, and compare to index funds.
That's not the comparison you have to make here. Instead consider :
price of the lot + compounding (regular income from utilizing the lot + rent you would have otherwise paid)
As it happens, I'm an officer of a family business that has a number of real estate investments (some residential, some commercial, some industrial). We've had the properties over a timeframe of decades -- in some cases like 10-20 years, in other cases 30-40. We're now selling them.
Some of them have been basically good investments. Others have been iffy. You do of course accrue rents from them, but there are also costs. There is no law of nature that they outperform other investments.