In the past three months I've lived in Palo Alto, SF, and now Menlo Park, and have been to just about every stop on the Caltrain from SF to San Jose.
SF may be easier to get along without a car, but I don't find SV particularly difficult. Some combination of Caltrain + Zipcar + Uber + taxi + bus + bike + short walks is always available.