You are not helping people because of their race, so why use that to select who gets help? There are rich black people as well as there are poor white people.
At the same time, the poor white person benefited from that original wrong, and you can isolate, statistically, some economic advantage to him tracing back to that original wrong. It doesn't matter how poor he is, he would be incrementally poorer had that wrong not occurred.