Add in the mortgage interest deduction (this raises prices because people can spend more than they would if they had to pay the tax on the interest); add in idiotic zoning laws / parking requirements in most urban areas of this country (40% of the buildings in Manhattan could not be built under the city's current zoning laws https://www.nytimes.com/interactive/2016/05/19/upshot/forty-....) and viola, you have super high housing costs.
The problems are easy to fix. Places like Tokyo have low housing costs compared to other cities of the same density. It can be accomplished, it's simply a matter of IF we'll do it. Given that homeowners and landowners have a gigantic portion of the assets in our world economy, I'm not hopeful that they'll ever willingly lower the price of assets for the good of society.