Like you, I've put it into words many times with my family. I have my cool friends, but they're turned off and intimidated by intellectual conversations and they don't have anything useful to contribute anyway. On the other hand, my geeky friends consider some of my interests superficial.

In my dream world, I'd have friends I could go lift weights with, have a BBQ, get dressed up, go clubbing, and talk continuation/passing style and credit default swaps while we do it.

I don't think you should communicate these issues to other people. Which group wouldn't be offended? It certainly won't help anything.