I would add that this will vary by industry and location as well. If you're in the bay area interviewing for a startup, you can get away with a lot more casual dress than if you're trying to get on the IT team for a financial firm in downtown Chicago.
One approach you can use is to ask the HR person over the phone what the culture is like, or to look at their "about us" webpage. These will provide other useful signals as well.
I found that if you're interviewing for anywhere in finance sector or public sector a formal suit is a must. For a 'tech' company a business casual(so a khaki and a button-up long-sleeves shirts with a base white t-shirt in) has never failed me yet.
On the other side of the table, when I conduct engineering interviews, I will notice what you're wearing. But don't worry. After interviewing over 100 candidates in person I've never turned down anyone for their attire. At most you'll get a nice comment if you look good and the rest of the interview continues. Maybe we'll joke about how you're the best dressed person I've seen all day.
For suits, please don't wear them if you can't make it look right. Ask your loved one or friend for honest feedback. I'm not going to turn you down for a position for wearing a bad suit, but I do worry about how others perceive you if you wear that suit in other events.
a) As a programmer, I'm not being hired for my appearance, but for my ability to get stuff done and create value.
b) I can get far more work done and think more clearly when I'm comfortable.
c) If you have a problem with somebody wearing jeans and a t-shirt or gasp, even shorts in the summertime, that's on you, not on me.
On the rare occasion I've ever had to go to a job interview, I've ranged from shorts/jeans to business casual at most and have never felt out of place.
If you want to play dress up for a living, go become a lawyer or model or real estate agent or some other profession where you're constantly judged on how "professional" your appearance is. That's not what programmers are usually about or motivated by.
So what you wear is completely irrelevant? Or does Martha Stewart a better security analyst than Bruce Schneier? With no disrespect intended to Dr. Schneier, she certainly has better decor, food on the table, and scents in the air.
If the weather is cool enough, I wear jeans, a button down shirt and a blazer when interviewing or meeting with a client the first time.
I did interview in gym shorts once but that was more of a formality.
Though I have a memory that my last interview I might have had a plain black fitted t-shirt instead, due to the lack of notice involved.