I live right downtown in Toronto, and while I have a car, I like to walk or bike places. I managed to find a great office that is a 5 minute walk from where I live and it is situated about 500 m from two major subway lines, close to the biggest university and one major college, a hop-skip-and-jump from the museum and art gallery, and countless other great places. I walk out my front door and within 200 m is a great café, a patisserie, a pub, health food restaurant (with food that actually tastes good), etc. I know more of my neighbours here than I did when I lived in a Toronto suburb.
I don't think the key is to live in San Francisco. I think the key is to get the hell out of the suburban hell holes that riddle North America like cancerous spots on an otherwise healthy lung, and find cities in which you can live that offer culture, art, helpful business communities (Google "Toronto BIAs"), understanding government, and diversity with respect to people and industries (e.g., don't live in a one-hit wonder industry city). I can understand that the proximity to the Valley is a huge selling point of San Francisco. You have to remember most of the people telling you to move to San Francisco are doing so because they want to invest in you and they can only do that if they can keep a close eye on you. If you're not looking for investment from those people, then live somewhere else.
I mean, there's some cool stuff coming out of Toronto. Check these guys out: http://www.ideeinc.com/.