Well we can't guarantee 100% fairness right?
You are right about the unconsciousness (yeah, I remember this from my psychology 101 class :) those famous prejudice experiments in the 40s, 50s).
My argument is once you are conscious of doing "I must hire more said gender, said race", you are sort of discriminating people who are believed to be minority. First, do they like to be called minority? Should they be given extra bonus point because they are minority?
Of course we can argue all day and make different situations when and when not to hire a minority over a majority, but let's not.
Granted, what makes one "more qualified" than another candidate. See [1]. This is an old post by Stripe's CTO. One of that is Sunday Test and the whole point is to see whether people like to work with the candidate.
X is super funny and X is as skillful as Y. But what makes people think Y can't have humor in his/her own way and isn't more productive than X? Data we gather from interview can only show some data. But that's where we stop because if we go full investigation we would spend a year to interview every candidate.
Unless we can completely hide our identity from interviewers... we will always be committing the unconscious bias.
[1]: http://www.quora.com/Stripe-company/What-is-the-engineering-...