Working in a startup is also chaotic, unpredictable, and fast-paced, so it seems natural that people drawn to startups would also be drawn to cities.
This isn't new either: there were plenty of SF startups during the old dot-com boom as well. I'd also predict that if there is a further growth of tech startups we'll see startups spread to other areas now considered "less desirable" -- in dot-com days there were plenty of companies in Santa Clara, Milpitas, Downtown San Jose, Fremont, and even exurbs like Pleasanton. When the boom ended, CA-237 was locally called the "dot com graveyard".
As you mention there is plenty of cheap and large offices as you go south along the peninsula, but startups seem to consistently choose more expensive SOMA offices over these. Only startups that need a warehouse or other specialized facilities start their search in the south bay.
I think the reality is that its a combination of factors driving people to SF, from 'cool factor'/perception to quality of life issues (employee proximity) to 'cheaper-than-palo-alto' benchmarking.
By the way, if you're reading this thread and looking for office space, the south bay is a perfectly good place to look. Yes, it's not as glamorous, but it's both functional, frugal, and actually quite nice. The graveyard doesn't consist of dumpy factories, but pleasantly maintained offices with good facilities. You can even live nearby, as long as calm neighborhood is an acceptable condition for you.
I would say significant amount of startups would still prefer Mountain View to SOMA, but they'd probably prefer SOMA to Santa Clara or San Jose.
There are many positives you've described to places further South, but there are also issues: fewer other startups, less third places in walking distances, being rather far from any kind of public transportation. In terms of employees it's somewhat of a balancing act: there are older employees with family in South Bay, more infrastructure folks (ex-Google and VMW folks, as well as various infra startups), and there are also younger employees, front-end developers, designers, and people with finance/consulting who live in SF. It would be more far more difficult for SF set to commute to, e.g., an office park in Santa Clara than for folks in South Bay to commute to SOMA (fairly easy trip by either car or Caltrain).
That said, Peninsula is exactly what its name implies, and SF is at very end (with hills in between). There will be greater pressure for startups to go further South again, as well as East (if Berkeley/Oakland adopt more startup friendly laws), elsewhere in SF (already happening), and so on.
You should visit Sinagpore once.
This is often not true around SF. Up and down the peninsula, you'll find continuous cities with strong tech and other industries. Would we consider Redwood City a stereotypical suburb, for example? It becomes more murky, and many of the stereotypes fall apart.
That said, your perception is held by many people and is accurate for many parts of the US. As a result, people moving to the Bay Area choose where to live with a similar bias.