But it has gotten harder in some important ways, housing costs as a % of median income have risen by multiples since then. Shelter being thing that probably makes people feel the most insecure
Those 'other things' are generally more optional than shelter, so you'll bid up shelter until you get it while sacrificing those other things. Inelastic demand and inelastic supply is a bad combo.
We see affordability and population growth in places that allow housing.
We don't see population growth without immigration anywhere but third world countries any more, and consistently dropping there too as living standards increase.
While I agree it's a bad combo and could be better, there's nothing to support any notion that cheaper housing will be enough to increase fertility rates.