When housing becomes an investment, the goal is to get that sweet ROI which generally means working to restrict the supply of housing to make prices rise. Lobbying is a high-ROI activity.
And housing doesn’t need to be expensive. Japan is a good example. It’s only expensive because we instituted policies that make it an effective investment vehicle when it’s a necessity in life. Many cities outside the US have policies in place making property taxes very expensive for an investor who doesn’t live in the house. In particular many countries prohibit foreign investors from purchasing real estate without a local intermediary who is legally responsible for any shenanigans.
The free market works well for many things, but there are areas where the motive to create artificial scarcity create some dystopian outcomes (see also healthcare). We have to ask ourselves as a society if our belief in the free market is strong enough to accept the societal issues (crime, mental health, etc) that have been proven to come along with unstable housing.