But then jobs in the bay area became much less concentrated in downtown SF, and those massive parking lots in faraway BART stations went underutilized.
Have you been to a BART parking lot recently during a weekday morning? Even the stops furthest out on the PBP line have nearly completely full lots.
I think the killer combo is self driving cabs for first/last mile, plus rail for intercity travel. Push a button 15 minutes before you want to leave, a cab sits outside your home for up to 30 minutes, you get a warning before each transit sync moment, hop in the cab, get dropped off directly at the platform, get picked up at the designation platform and then dropped at your destination.
It's basically optimal in terms of carbon, throughput, and convenience. Maybe total transit time is no better than a personal car, but you could punch through traffic so at peak times it would be better. And with high speed rail, it could beat personal car even in off peak... At least for some geographies. Obviously if you're not near rail it won't be an improvement in Transit time.
If I'm not mistaken you are basically thinking of a circuit-based network (with ad-hoc 'datagrams') while Musk seems to be going for a completely datagram based network. It's an utterly genius idea, at least from a theoretical standpoint (I have no knowledge of the economic aspect).
Millions of tech nerds daily grinding through a hellish commute so they can spend the day sitting in front of a computer. 99% of them doing nothing that they couldn't just as easily do from a computer at home.
Clearly optimized transportation is the best solution here.
Something something... better horses.
There's currently 4 year waiting lists for many of those massive parking lots in faraway bart stations.
Even if a really far-flung station is, say, 3/4ths utilized now, won't the next decade or two of housing development create more users to fill those spaces?