I get what you are saying, but sometimes it's about narrow streets, sewer systems that are at capacity, water mains that are too small, traffic, horrid transportation systems, just towns that weren't built to scale?
Yes, you will find groups of people who don't want new people moving in, or the town they founded spoiled with overbuilding.
Bolinas comes to mind. These guys(Basically Hippies) built houses out here when no one realized just how beautiful a small community can be, and just how beautiful it is out there. They designed their little town with only a certain number of water hookups. They were upfront about their desires--they didn't want developement, nor other people. It essentiall stopped all building. As a young man who wanted to build out there, I thought it was unfair; but I can also see the wisdom in what they did. It's just a complex problem.
I am sympathetic to people who need housing. I actully belive in rent control. I just don't have a answer.
On a slightly different note, I belive we need to end homelessness yesterday. Every town should be required to have an area where people can camp. It doesn't need to be fancy. Just a place where a person can pitch a tent, without getting a ticket, or killed. There can be porta potties, and mobile wash stations. Cops can patrol the camp sites if needed. I don't think everyone will want to live in these primitive campsites. If that became a problem, you could require proof of previous residency.
Got off track--sorry.