I really don't understand what the distinction he's making here between "nerds" and "smart people who aren't nerds" is.
All smart people, nerdy or not-nerdy, have a vast collection of random trivia stored away in their brains somewhere (dumb people have something similar but smaller), and everybody enjoys saying funny things. What's the real distinction here?