But for a brief 40 years before the super-constriction of supply happened, it was a welcoming place to lots of people that couldn't find homes elsewhere, because there were at least some empty places to rent out.
What drives up the prices isn't that people with big salaries are there, they've always been there. What's driving up the rents is that the people with big real estate holdings have said that the people with paychecks can't build anything new. So instead of those big paychecks going to building more infrastructure and subsidized low-income housing, those big paychecks are going only to current landlords, and pushing out anybody without a big paycheck.
SF housing is a game of musical chairs, and all the people that found their chair in the 1970s and 1980s easily because there were lots of empty chairs have decided that there should be no more chairs. That provincial, close-minded, and ultimately xenophobic attitude is what makes the housing market so unfair.