First thing I looked for was, "Bart around the Bay" - nothing there. It's beyond me why nobody has made getting Bart circling the bay a priority, and really is a blight on "Bay Area" transit.
Caltrain is not an option because the service isn't 24x7, it's terribly slow, the train doesn't come often enough, and the station isn't close to where I live. The last 2 are just personal annoyances, but I think the first 2 are enough to make it a much-less-desirable option vs driving for people who would otherwise take it. That the cars are ridiculously over-built because they technically share tracks with freight trains (from what I've read, this is also what makes them so horribly loud and inefficient [which also makes the system much more expensive to run]) and do lots of PR damage by having very loud horns (my speculation: is that also related to the shares-tracks-with-freight-trains thing?) are just rotten cherries on top.
Public transit isn't easy, but it's also not impossible. I know that even the Bay Area doesn't have the population density of places with their public transit
(Santa Clara decided to spend that money on those silly expressways instead.)
Where would a BART line run on the Peninsula? Down 101 from Millbrae? Such a route would cost billions over some of the most expensive real estate in the country, yet wouldn't serve the denser parts of the peninsula well.
Modernizing the Caltrain seems like a much better use of money. It's clear from the website that this stuff is very expensive!
If we want to dream, a BART extension from Milpitas through Santa Clara (and the new stadium) to link up with the light rail and Caltrain in downtown Mountain View would be awesome, though!
Caltrain is WAY overdue for upgrades. The electrification project, which will solve some but not all of these problems, is in need of a speedup. Many at grade crossings need to be replaced with bridges (and some are being replaced), but because of all the fiefdom-like towns along the way, getting anything done along the right-of-way is a nightmare.
Although there is no tractable way to merge Caltrain with BART (because of BART's own foolish planning decisions, such as a custom gauge), and you're right that there is no space for a second right-of-way, there is another neglected project that would benefit here: Dumbarton Rail (http://www.bayrailalliance.org/dumbarton_rail). At various points it was suggested that either BART or Caltrain should be extended across that bridge - either way, completing that link would add badly needed capacity in a region that has long neglected its traffic infrastructure.
As a serviceable mass transit system, it's almost worthless. The trains come infrequently, and require really inefficient transfers in the city to BART if you want to travel from, say, Mountain View to Oakland.
I just came back from Singapore - and I was able to get almost any place in the country + a 5 minute cab ride the entire day using their MRT. I understand that Tokyo (and to some degree, New York) have similarly advanced transit systems.
I used Caltrain + Bart for about 15 years, but you really don't realize how bad Caltrain + BART are, until you travel somewhere that was a first class mass transit system.