NYC seems to be on par with SF. Both living situations are extreme for someone with a more suburban upbringing.
Lack of mobility if you don't have a car, lack of available parking if you do, long lines in general, lots of traffic. Very poor care at many establishments due to overworked and over stressed staff. Miniscule and generally not so great living conditions. For example I lived in a nearly 100 year old house in Oakland with others, which 1 yr later found out had mold problems (a serious health hazard). Also lived in two very modern apartments in the heart of SF both of which had a converted living room bedroom with three roommates. One roommate slept on a couch for a year, paying over $1000/mo. Crime due to homeless situation (witnessed crime frequently during my time, petty crime especially. Side note, didn't a Bart train get robbed just yesterday?). Medical care is another big one (multiple experiences, one for me personally)
Even little things like cooking are very difficult when your entire kitchen is 1/5 the size of what you'd find in a similarly priced suburban home. Little things like this add to stress as I wrote a bit about already.
And sadly the south bay isn't that much better as a "suburb". The home prices in Palo Alto for example are quite outrageous. There are a lot of far nicer suburbs in other parts of the country without the wild price tag.
I was young when I moved, and don't regret the time I spent in the Bay at all. I learned a lot, it was positive for my career and most of all I just had to get it out of my system. The glitz and glamour of it all wore with time, as I suspect it does for many, and ultimately I felt it wasn't where I would want to build my future.