Currently it's almost an hour to get to downtown from Ocean Beach to Financial district with a bus, and its only 7 miles away. That is 7mph, almost a walking speed.
For example, back in Helsinki (where I'm from) you could easily get to the downtown anywhere in the 12mile radius in 10-20mins with a bus, light rail or train. I know the numbers are similar in many European cities.
Generally in Europe public transport is usually the easier and faster option than driving or at least the same time. If driving in a city is multitudes faster then you're not doing to public or other transportation systems very well.
The only politically realistic way we're going to get large-scale high-density environments is through greenfield projects. Having a long but tolerable public transit journey between your greenfield high-density development and the existing job centers is one of very few plausible paths to getting there.
You need sensible building policies coupled with aggressive transit expansion to get us out of the mess we're in. Not just in San Francisco, but across the whole country.
Evaluating this proposal by itself, I agree that local infrastructure spending would likely be more useful, sooner, to more people than a high speed rail corridor. I don't think anyone believes or claims it will single-handedly solve the housing crisis.