You forgot to mention zoning law which restrict the supply of housing a lot, much more than any wheeling and dealing cities may do with developers.
Supply is simply not allowed to increase in many areas. It is not a free market, it is government controlled and the priority is to preserve/increase the property values for existing home owners, and restrict where people can live by income. Talk of affordable housing does not come anywhere near making up for the restricted supply in cities due to zoning.
There are other countries that have a much different approach to zoning, there's ideas like land value taxes, etc.