If I had gone to Harvey Mudd or Rose-Hulman, and my interviewer hadn't heard of my school, I might have backed out.
Also, after a couple of years of work experience (after your second job), nobody really pays attention on where you went to, so don't get too hung up if your interviewer is not familiar with your school.
It isn't the name, but the strength of the program that matters.