The problem is that Caltrain (and BART outside SF proper) has its stations along the periphery instead of the heart of town. You can't jump on Caltrain in the Mission or Noe Valley or even Market St, and on the southern end, it's not going to drop you off anywhere near anything.
This is because California, and the Bay Area in particular, follows a policy of "The needs of the few outweigh the needs of the many", and when previous generations were deciding where to put stations, they didn't use eminent domain like most municipalities would; instead, they built them either in the few parcels of vacant land off on the periphery, or along the freeway land they already owned, which is perhaps the most pedestrian-hostile arrangement possible.