Housing needs to not be an investment; it needs to not be /attractive/ as an investment.
Maybe that means massive taxes on the property if the owner isn't using it as a residence for at least 1/3rd of the year (to allow for snow-bird dual-home setups that are popular).