If society thought of housing similarly to other large purchases like cars in that there was no expectation of profit, we would all be a lot better off.
Now what caused this housing as investment idea? I haven’t seen any research on this, but my guess would be as a form of forced retirement savings as it allows people to build net worth without the self-control normally required of saving. And the difficulty of that delayed gratification is really just human nature.
And you don't get margin called.
150x leverage means 150x cashflow burden from taxation (> 100%!) and also maintenance costs of the property.
See https://news.ycombinator.com/item?id=35044028 where I laid out how a real estate purchase is really a mix of multiple investments packaged into one sale.
To what extent the crime policies contribute statistically I don't know, but for some of the more extreme mental illness cases (that disproportionately account for homeless people being actively aggressive), there is a disconnect between philosophical ideals and what practically would be better for everyone in the long run. Mental illness (especially when substance abuse is involved) can make someone resist help and they will be unlikely to improve unless they are forced into a facility. Where the line is for doing this is a tough question but if someone is routinely threatening people on the street or does something violent, it seems like SF policy is still opposed to it.
I don't think people realize how much of an outlier SF is. Incomes are a good 25% or so higher than NYC and Boston. The percentage of the people who work in tech in SF isn't really comparably to any other city and their own high paying industries.
I think the problems with SF are largely cultural and political rather than technological or economic, though the economic issues are exacerbating it. It's an "anything goes" city and has been for 150+ years. The city values your ability to express yourself in any way you please, and that includes both healthy forms like street fairs, neighborhood scavenger hunts, or Bring Your Own Big Wheel, and less-healthy forms like being able to shit on the sidewalk when you please or stab someone who looks at you funny. And that filters down to the voting preferences of the electorate. You get what you ask for.
The South Bay has all the same problems of tech dominance, income inequality, insane housing prices, etc, but the cultural orientation is very different. People in the South Bay are generally there to work hard and build a better life for themselves and their descendants. Their time-orientation is much more long-term than SF culture. And that applies across the income spectrum - even poor, Hispanic, immigrant day-workers will do their best to make sure their kids do their homework. The type of tech industry that spills out of this also reflects this cultural orientation, with the South Bay doing much more foundational engineering research and SF being more focused on how to apply technical breakthroughs from 50 miles south in new and crazy ways.