This excess for investing doesn't exist. Its in fact cheaper to buy in SF every month then it is to rent.
Trulia has a really cool tool where you can mess with the numbers yourself
First $6k/month will not get you a $1.5MM home unless you had a down payment of $500k and even then it would not cover your taxes and other expenses. A general rule of thumb for someone with good credit is a monthly payment of $3k will let you borrow $500k on a 30-year fixed mortgage. You can check this yourself here: http://www.mortgagecalculator.org/ (or use Trulia's).
The problem with Trulia's "really cool tool" is unless you use the advanced version all the inputs are wrong. For example, its tax assumptions are less than half of real cost for SF. A more sophisticated tool is available from the NY Times (http://www.nytimes.com/interactive/business/buy-rent-calcula...).
Of course getting an apples to apples comparison is pretty hard unless you're talking about buying a condo vs. renting a condo in the same building. Yes, it would be cheaper to buy a 1 bedroom in Bayview then rent a 3 bedroom in Pacific Heights. If you're looking at starter homes in a decent area, say, Noe Valley and compare those costs to renting something similar in the same area renting is generally cheaper.
The "excess for investing" DOES exist, at a minimum in the form of the down payment you did not spend. The good news for home buyers is that you get leverage on that return if the market keeps going up. The bad news is that it's illiquid and we don't know whether that's a better investment than the stock market or one of the companies we see on the front page everyday. In short, if you're never moving again buying is likely cheaper. Otherwise, it's complicated.
TLDR: It's complicated, and most calculations downplay other buyer costs and ignore other renter opportunities.
1000 sqft loft: $3300 1150 sqft house w/ 40x50 yard and 1 car garage: $2725
But, that isn't in san francisco, but it is within a similar commute distance.
Out of curiosity, how much would you estimate it costs to buy the house? Some quick math will let us figure out the rent vs. buy for your neighborhood.
My place was $470k and we paid about $72k in closing costs. Our loan is 3.625% for $417k.
You can still find nice places at that price point, but rates have gone up. So that's another reason why it isn't a fair comparison. But it's an option, and it isn't far away like other east bay places. And like I said, commute times are similar (35 minutes for me).