The more interesting question is, why are the 50 miles between SF and SJ reserved for sparsely packed single family homes and one story strip malls?
Redwood City is also nicely positioned as a mid-point between SF and South Bay.
Beyond that, it seems the entire El Camino corridor is being heavily developed with retail and high-density condos and apartments.
Some of these areas (Redwood City in this case) are next to major Caltrain stations with Baby Bullet stops. But they're also convenient (technically speaking) for going on 101S/N to work, etc.
That's part of why I'd prefer living on the peninsula than in downtown SF. Weather is consistently better in the summer months. More space/less people (except for rush hour traffic). More parks for recreation. Pools aren't necessarily a deal breaker for me, but it's a nice extra.
A shiny high-rise is nice but unless you can afford a penthouse or similar you're probably looking at a small unit. Less space, fewer parks, more people everywhere (tourists, homeless, etc.)
The downtown has a solid layout though and lots of room for growth. The RWC downtown has the potential for offering a lot more than some neighboring downtowns that are less dense and could be a nice hybrid between SF and more "suburban" Peninsula cities.