Or at least a regional authority. Something like the ABAG but with authority and teeth. Otherwise we end up with local governments doing their own uncoördinated thing, so there is lots of duplication and also lots of haphazardness to development and organization, and perhaps most pernicious, lots of localization.

That's true until residents start to accept payoffs to defect, which is what's happening in SF when wealthy outsiders pay above market to get their hands on property.

Residents can do nothing and just let that keep happening, but if they do, they'll eventually end up getting priced out.

The opposition to density is actually happening more at the neighborhood level than the city level. Current zoning rules would allow quite a bit more density in many neighborhoods, but the approval process allows for tremendous delays if neighbors object.

It's true that this could be fixed at the city level by streamlining the process. My supervisor, Scott Weiner, is pushing that, specifically for 100% affordable housing. That's not going anywhere, which demonstrates that the opposition to development is more about wanting parking, light, and views - not about affordability.

Unlike the rest of the country, there is no right to build in SF if you meet all zoning requirements. Zoning isn't even the problem.

San Francisco is largely made up of renters (63%) and planning decisions are made by the City of San Francisco, not individual neighborhoods. The problem you describe exists in small communities where the neighborhood is de facto a jurisdiction. Like Brisbane, right south of SF. In San Francisco, NIMBY homeowner are actually a minority playing against the interests of the majority of the citizens of this city.

Now there are several causes for this. A major one is concentrated harm vs distributed benefits for specific development. A new development in a residential neighborhood is gonna piss off a small number of people and motivate them to organize to put pressure on the planning commission. This development would benefit pretty much everyone else, but in a way that is so distributed that it's really hard to motivate people to fight for it. A new development is gonna have a very concentrated impact on neighbors and a very diluted effect on the rest of the city.