I haven't traveled a lot (~18 countries) but I've not encountered any poverty in a first-world city that is comparable to SF. Sure there are homeless in other cities (e.g. Rome's beggars) but not with the same confronting mix of desperation and mental health issues that I encountered in SF (I've been told NYC is worse, but I haven't been).
What exaggerated this reaction was that SF was my first US exposure. From an outside perspective we're talking about a nation with the world's largest economy since the 20's that prides itself on being a bastion of success and freedom.
The social reaction was likewise confusing. Avoiding the center of a modern city because law and order cannot be upheld was an unfamiliar concept to me, but the fact no one (police or civilian) seemed bothered that there was effectively a "no-go zone" in the middle of their city was even more surprising.