I've lived in all of Durham, San Francisco, and Menlo Park (next town over to Palo Alto).
Durham, in my mind, had a weird vibe. Despite growing up there, I think what I remember best is taking a swing dance workshop (in my mid-twenties) from Sharon Ashe. Sharon had been at the heart and center of the San Francisco lindy hop community, possibly the best swing dance scene in the world. She had co-founded the 9:20 Special, which still attracts a crowd of hundreds every week.
But apparently she had burned out, and had too much of the cost of living and the competition (there isn't really any smog) and decided to move to Durham. She wanted a lower-key, quieter life, while still enjoying most of what she'd enjoyed in San Francisco. At the time, this struck me as a profoundly sad thing to do, and soon after I left town.
I would say that compromise is at the heart of what Durham has to offer. This is ultimately why I left, but it has got great restaurants, good weather, a lot of culture going on, and above all a wide range of diverse, interesting, active, intelligent people. Just not quite as much of any of the above (except for the weather) than SF or Cambridge. Take that for what you will.
e-mail in profile if you have questions.