I hope we get more offices opening up in Vancouver soon as well. Toronto has more momentum as the tech centre but Vancouver has better weather and is a beautiful place to live.
My spouse is a case worker in the housing department of an Ontario municipality so I get all kinds of inside information on the subject.
You're not gonna personally get yelled at/followed every week, but confrontations are not rare. I know several people including myself who have experienced it a few times.
There's a huge, poorly addressed mental health issue in Ontario, and downtown Toronto concentrates a lot of that.
Tangent: Coming from a third world country, homelessness in North America has a huge contrast with South America: my own anecdata is that mental health issues among the homeless are more common in NA and they're very irate and confrontational. I often wonder why there's such a contrast.