I think he's not necessarily arguing against t-shirt and jeans (despite his claim that the "days of t-shirt and hoodies are over" - this may be an east coast thing?), but that fit matters. Just because you're writing code doesn't mean you have to wear the ratty black t-shirt that you haven't washed in weeks and the ripped jeans that you bought in high school.
I know that, like with the "frat boy" you described, there are also stereotypes with the stereotypical programmer in the black shirt with the stupid text on it. How do those people make you feel?