Definitely about the people. Unless you are the founder of a company, you are never working for the _current_ company, but instead are working for the next company (in which you will be a founder, or an early stage employee that gets a good chunk of equity) - you need to demonstrate that you are someone they want to hire.
With the possible exception of craigslist, where they manage to accomplish heroic feats with very little in the way of staff, any company that is eyeballing $1B+ market-cap will grow to hundreds, and possibly thousands of employees.
Where else on earth can you find such a concentration of DBAs, Sysadmins, Designers, Tech-Friendly-Lawyers, Developers, Product Manager who have all three of the following characteristics:
o Very, Very good at their job.
o Real world experience in several major companies.
o And, most importantly, willing to go to work for a "lets throw caution to the wind" startup.
Change is the only constant in the valley. People here, for whatever reason, are just comfortable changing jobs, sharing information, and building new things.
The weather is astoundingly good. Never too hot or too cold - 350 / 365 days in the year. And the other 15 usually mean having to actually turn on your heater or open a window on a warm summer night.
The peninsula, for the most part, does suck in terms of lifestyle. There are exceptions (Murphy Street in Sunnyvale, Good portions of San Francisco, downtown San Jose on a Saturday night) - but, in general, it's mile upon mile of suburbia and strip malls. This is not Manhattan or downtown Vancouver. (On the flip side, with few distractions there is more time to get work done)
The major downside is that unless you are filthy rich or already a home owner, the only sane strategy (unless you don't mind commuting for 90-120 minutes each morning) is to rent.
With all that said, if what you are looking to do is grow a company organically, and have your eyes set on something more sane like a $10mm-$100mm market cap over 5-10 years, there are probably a lot of ways to succeed without being in the middle of it all.