It's going to be one diverse person at a lot of those boards and they will get the stigma of being the person that was put there because the actual good candidates had the wrong skin color. No matter how qualified they might actually be, they can't escape the looks.
Plus, by making this a factor, you're just kindling actual racism by pitting white males against everyone else. The resentment created by rejecting someone for their gender or color of skin does not subside just because the attributes happen to be white and male.
I get the intention, but fighting racism with racism just doesn't work.