This ignores massive push for people moving due to work, so in better places there is exponentially more interested people than space available. Just look at SF or NY is we talk about US.
Plus add strong emotions, most world population simply wants to live in their own, renting is frowned upon, despite ie Switzerland having very low home ownership, population is one of happiest globally and nobody here bats an eye, there are more important matters in our short lives to focus on than gathering as many available resources in some rat races.