The long and short of all of this is we are giving person A an advantage over person B because person C did something bad to person D, and it just so happens persons B & C have some matching demographic factor.
Putting it that way, I think the racism / sexism of diversity is laid clear. I don't call it reverse racism because racism can go in any direction it's not a unidirectional graph pointed away from white people. An innocent person is catching the short end of the stick as a dark twisted way to right some wrong that this innocent person had nothing to do with, other than skin color. The sick joke of it is that the ameliorative steps to fix racism are also racist.