Well, back in the day, "nerd" meant "social misfit". Which a lot of us were (me included), because we were too busy trying to be smart to be worried about learning how to play the social game. (Paul Graham wrote an essay about this, "Why Nerds are Unpopular".) But in the real world, it turns out that social skills also matter, and I should have realized it earlier.
But these days, "nerd" means more "socially awkward but really smart", which is neutral to positive from our perspective.
"Brogrammer", on the other hand, to me at least means "male who thinks he's really smart (even if he isn't) and therefore that he can be an offensive jerk to others, especially females". It would bug me a lot if someone thought that was true of me.