As someone with a DOD-heavy work background I have experienced firsthand that kind of arbitrary rejection.
I have a lot of friends with plenty of DoD, NMCI stuff... but you know, they show up to an interview for a digital agency in full dress with a perfectly pruned resume and pressed shirts... because that's the culture they are used to, and it gives off the wrong vibe. What it shows, to the other side, is that your priorities are in those places... that you see those things as important... and its a turn-off to places that are more fluid and open. I wouldn't work anywhere where I had to wear dress shoes and a collard shirt every day. It would be a BAD culture fit, because I don't value those things, as I see absolutely no connection between my clothes and my output as a developer...
One of those friends with heavy DoD/Corp background, i helped him get his job, he loves it now, because hes good with the more relaxed culture at the agency he's at now... but it was like pulling teeth to get him to show up for an interview not dressed like he was going to a wedding...
While I agree that the way someone dresses has no correlation with their developer output, personally I wouldn't give someone any shit about dressing too nicely for an interview that happens to be at a startup, especially since they're probably used to "playing it safe" when it comes to interviews. I'd probably just make it a point to point out typical dress code when the interview process is coming to a close, if the interviewee hasn't figured it out by then.
... or that you were taught it was just the way the world works. You can't tell if someone is doing it because they think it is important or because they don't know any better and want to err on the side of caution. Yes, to some degree you should learn about the companies you want to work at and should be able to figure out that showing up in a suit and tie at a webdev shop is probably not necessary. However, there is so much conflicting information from careers sites about what is proper that I cannot fault someone for showing up dressed like that.
My other issue is that you are showing the same type of judgment and bias based on appearance that the people who set the pressed collar dress codes do. To them, showing up in jeans and a t-shirt shows lack of attention to detail and appearances that they believe translates to your work. Personally, I couldn't care less what people look like in situations where appearance isn't important, as long as they are publicly decent. You want to show up in a three-piece suit? Fine. You want to show up in shorts and flip-flops? Fine.
Your attire is part of your ritual. If your work day begins with a cup of coffee and a conversation with some friends in the office, and you skip it for a week, your week feels off. Because your ritual is off.
If there's no connection, then why does it matter how the other person dresses? They're an employee. Unless you're hiring them to hire everyone else, in which case it may matter because you need their judgement to match yours.