Housing prices would come down (although many sellers would back out, crushing supply), but mortgage payments would skyrocket. Interest rates and housing is kind of like squeezing a balloon. Building new homes is like letting air out.