Isn't it though? I think that's the point: a well-off or even rich black kid from two black lawyers or doctors or whatnot is being favoured over a poor white kid from a (former) coal mining community in West-Virginia. The number of university openings, job openings, grants, etc. are all finite in number, so there is some amount of "competition" involved.
And you can talk about statistics and averages and whatnot all day, but once you start applying averages to individuals – which is essentially what "positive discrimination" or "affirmative action" amounts to – you're always going to be wrong in a very substantial number of cases.