After about 5 years, I realized that no matter how much vocabulary I learned, how perfect my grammar was, or how reduced my accent was -- I would never get the references to childhood cartoons, or any of another 1,000 things that came from growing up there. And it's actually crazy how much of casual conversation revolves around all these things.
And finally coming back to the States, it was all just so easy -- such a luxury to have a whole shared cultural history with people I'd hang out with, so many jokes to make, so much richness to the conversation. Social conversation felt "full" again.
That's what they never tell you when you learn a new language -- the language alone isn't enough.
Addition: the funny thing is, I have a lot of non-American friends who get all (or most) of the American cultural references, because so many of them revolve around TV/movies, and those are exported around the globe. So part of the American so-called "melting pot", I suspect, is helped by the worldwide ubiquity of our entertainment culture.