Most "buildable lots" tend to be in subdivisions ... which already have HOAs. "Infill lots" tend to either be in terrible neighborhoods or amazing neighborhoods that are out of reach, budget-wise, for most.
Building on a rural "plot of land" is often more restricted than you'd think, due to zoning regs on minimum lot sizes and rules about water meter availability. And frankly, buying and caring for 10 acres puts the whole project out of reach for most.
Not to mention internet availability is problematic as you get into rural locations with more flexibility.
I don't necessarily want to live in one, but there are essentially no houses built in the last 50 years in my area for sale that don't sit in an HOA.