Where in SF do we have shit tons of land? Building high volume transit would take a lot of land -- the entire BART system is constrained by how fast people can load/unload at Embarcadero station. The signaling system could push more trains through the tunnel, but the station can't handle more. The solution to that is to build more platforms. It'd be easy to do that with a ground level station or a shallow cut-and-cover station, but now it'd take digging out a huge cavern 50 feet underground while avoiding existing building basements and support pilings.
BART plans a billion dollar bandaid by digging out the other side of the station to allow boarding from both sides of the tunnel. But that billion dollars doesn't buy any additional tracks, just an extra exit from the sides of the trains.
Even the new 6+ billion dollar transit center is only going to have 3 platforms and 6 tracks for Caltrain and HSR. In comparison, Penn Station has 11 platforms & 21 tracks.