> rational is the fear of any new housing for homeowners in wealthy areas
That's the wrong question. The question is how strongly are irrational fears; more specifically, how strongly are the irrational fears of the minority of activist NIMBYs?
The understand the problem you have to understand how the process works and how its abused. The response to the massive urban projects of the 1960s and 1970s was to increase so-called community involvement. In a city like San Francisco, pulling a permit for even a minor project can result in hearing after hearing. There are people who dedicate all their free time to going to these hearings to raise objections. Objections need to be dealt with individually. If they don't then NIMBYs can go to elected officials to complain that community input procedures aren't being followed, or even go to court to obtain an injunction.
The ins and outs are complex and vary widely by jurisdiction. But the short story is that there's an extremely vocal minority who cause a huge amount of trouble by abusing process. The NIMBY vanguard can assemble larger opposition groups if a project moves too far along as they're adept at FUD (Fear, Uncertainty, Doubt). Though most people don't have strong opinions, they're naturally biased toward the status quo and therefore susceptible to extremist NIMBYs' FUD campaigns. They assume that when they see smoke there must be a fire, but in truth the extremist NIMBYs spend their lives blowing smoke. It's just that the only people who usually see it are the city hall employees and elected officials.
Agencies and developers know the drill so the process has gotten stricter and longer in anticipation of the community input phase. But it hasn't helped anything; it's just a symptom and part of the downward spiral of chaos.
Once upon a time poor communities bore the brunt of imposed development. But now there are professional community activists who have the knowhow and wherewithal to push back. Indeed, part of the reason for the more community-involved process was precisely to even the playing field. But just as before when community input was only a matter of holding your representative's feet to the fire, richer neighborhoods are more adept and more capable of using this process. So while poorer communities are better at pushing back, the end result is that there's simply less overall development.