I was surprised to find out (in 2009, may have since changed) that there wasn't a daily train between SF (and surrounds, pop 8M) and LA (and surrounds, pop 20M). I had to get a bus to Bakersfield before I could hop on a train.
That's mostly a problem with the terrain geography in the area (trains can't easily cross the grapevine without a tunnel underneath it or a much longer travel time).
You can take the train directly to Oakland, but not San Francisco.
My Bakersfield train ended up in Oakland, and I had to get the bus over the bridge to finish the journey. There didn't appear to be a train direct from LA to the same location. Again, this was a few years ago, so things may have changed. Or it may have been seasonal.