I'd even say I was stunted socially early on for not participating in much pop culture. Sports and music were important even all the way back in elementary school.
The threat is becoming another pondskipper on the ocean of human experience.
When you know 40-something people who've spent their entire existence revolving around popular culture, you know somebody with no opinions or ideas or feelings the box on the wall didn't package and deliver to them.
It's important to be able to relate to other people, but when so many people are consumed by something, religion, politics, 'popular culture', then I think it is fair to wonder why you'd want to relate to that, they look a lot like dead ends.
Revolving a good amount of your life around popular culture absolutely does not mean you don't have your own opinions. If everyone had the same opinion about characters on a show, then there wouldn't be much to talk about. Sure, not everyone is going to have their own opinion on if the latest P=NP paper has any legs to stand on, but who cares?
In addition, what's the difference between watching a high quality TV show or movie and reading a fiction book? Sure, the book may be longer and more detailed, but there are plenty of extremely high quality TV shows being produced now.