There's no reason for this, either. San Francisco has to relax their brain-dead backlash toward high density construction. Los Angeles, too. Is it a California thing in general?
Feel free to be cynical in your answer, but why would a property owner who just paid a fortune to buy a single family house in SF want to rezone the neighborhood to see a large apartment complex built next door? Now that they've paid the heavy toll, they stand to lose the most.
It's a vicious cycle. Some behavior has to break it.