Yes, there is this theoretical moral debate about what is right and what is wrong, but the real effect of doing what you say is right, would result in universities of only asians and whites, and the blacks mostly doing manual labour.
Is THAT right?
So it's okay to select mid western people, because they are different from new york people, but it's not okay to select black over white?
Those picky schools mix and match to have a wide array of people. They don't just use scores. If they did, there would be a lot of chinese immigrants and russians jews in there.
The difference between two people because of their geographical distance is as great as the difference because of their race.
MIT's big advantages have to stem from it being a less intense version of CalTech WRT math and physics. It simply can't afford to admit anyone who can't do the work (and there's a lot of self-selection in the applicant pool), doesn't do much with legacies (once again, they have to be able to do the work) and doesn't take many students from "fancy prep schools". So it's got lots of room for these students who are heavily discriminated against by the other elite schools in the nation.
And for science, at least, CalTech and MIT really have to make a big difference for those students. Everything in those fields depends on professor recommendations (getting into a good grad school and what you do after getting your Ph.D. and postdoc(s)). No matter how good you are, if your recommendations are from professors the targets have never heard of, you're ... not in a good posture.