Edit: As an addendum, I'm not referring to the "wrong kind" of people. I'm referring to too many people, or a facility with drastically different traffic patterns. Ignoring the "wrong kind" of people thing entirely, I could knock down a few suburban houses, and replace them with super-luxury furnished condo-townhowse-type things that would sell for the exact same as the surrounding neighborhood. This still would be a major change to the community, both in appearance and lifestyle.
They don't have any power in the zoning boards. 100 people could want to buy and build high-density housing in Palo Alto, but 10 people who already own property there could defeat their intentions via the zoning board. There is no reason why the desires of the few should outweigh the demands of the market.
If you don't want to live around high rises, buy up the land around your house. You shouldn't get to achieve the same effect by preventing other land owners from developing their property as they see fit. The zoning laws that allow this sort of behavior just favor existing property owners over future property owners, and lead to globally non-sensical development choices.
> If you don't want to live around high rises, buy up the land around your house
How does this scale? Here's a scenario: 100 people who don't want to live next to high rises but are perfectly fine living next to other single-family dwellings buy up 100 acres of land in LibertarianUtopiaLand and chop it up. They then sign a contract amongst themselves requiring community consent if any of them are going to drastically change their property, and require that any transfer of the land be accompanied by the same provision. Sounds quite familiar.
There's no national regulation saying that every community must use the same planning methods. It just so happens that there's a pretty good model for how the process works so implementing it requires less lawyers.