This is pretty much the crux of it. High housing prices could be "fixed" almost overnight, but for almost every citizen of the first world, their house is the single biggest investment they will ever make in their life. You'd be destroying everyone's retirement plans and there'd be major economic repercussions.
But there's also the second-order problem which is the people who realize that governments will never let the value of housing decrease because of the above, so you have upper middle class people owning "investment homes", or foreign investors buying homes they never intend to live in (or, in many cities, never even bother renting out), or even corporations like BlackRock buying up houses because they know it's such a safe investment.