To your last point - you know the untouchable caste in india? Imagine you have a poor untouchable caste, and you have a poor of the highest caste. You want to help them both, so you give them both some amount of money. Even when the untouchable creates some successful business, he is still untouchable.
That's the same problem you have - your society strongly associates black with negative. That stereotype will continue to exist, unless there is a proper movement of blacks into the standard college educated strata. Otherwise, they stay down in the 'untouchable' caste.
The child of a white person who was once poor does not have this stigma anymore. But the black persons child is still stuck with the stigma, because it's part of his skin-color.
That's the problem - your lower class is also the black class, and if you want to do things only the 'just' way, then it will remain that way for a long time to come.
And by the way, if you want equal admissions, then accept that you will not get into those schools because chinese immigrants and jewish people will probably get the best scores. So please, advocate merit based only or advocate that the school can choose, and accept that it can choose blacks.