It has the high density/high cost thing I dislike, and shitty weather. But has a lot of smart people, sane immigration policy, etc. (I would probably ignore a Toronto if it were in the US and otherwise the same, but Canada is a win for a lot of things.)
I guess Switzerland, Austria, Liechtenstein, Czech Republic, etc. go on the "potentially good" list, too.
The things I hate about SF are really threefold:
1) I've spent "enough" time here to be bored with it, at least outside work related things. It's genuinely not a world-class city (outside of tech, and food) -- LA, NY, London, Tokyo, Shanghai, Hong Kong, etc have it beat)
2) Absurd costs due mostly to demand and somewhat due to government policies -- this has been well covered. It essentially means you must always be working in tech, unless you've had a huge exit, unless you moved here sufficiently long ago or accept a sufficiently low quality of life.
3) Aside from costs, crazy government and social policies. I can get that people disagree on political issues, but SF (and CA as a whole) are just badly governed -- they collect taxes like a high-services-returned socialist state, but provide services like a low-tax state. There are a bunch of structural reasons this is unlikely to change, so it's not really worth investing any effort in CA politics. I'd actually prefer living in a place which is at the current time more fucked (say, TX, TN, KY, Indonesia; whatever), but where there's enough of a desire to fix things that the trend is positive. Or where the people are actually pretty sane and reasonable overall (Seattle; many of the Northern European countries).