I'm not sure I understand. If it is discovered, or "found" (by the media, I guess), that "these systems" are biased, I suppose that should result in them, being fixed, even at the expense of them "tak[ing] off." In your view, is that a good thing or a bad thing? Is it better or worse that "the media" find it? Also, by "normative historically," did you mean "representative?" In other words, do you mean a mathematical norm, as in an average? Or an ethical norm, as in a rule? (Hetero-normative, normative ethics, and so on.) Surely we're not talking about "correct biases?" It's just that my impression is that the language of "normative" in this context, a context which reads as sociological, is most often associated with societal, cultural, or ethical "norms," which are things that are viewed as correct in an "ought" sense.
> Just look at the contentions of prison reform automated systems and the like.
Can you elaborate? Because I think very few of us have looked at these examples. Which "prison reform automated systems and the like" have failed to "take off" due to a discovery of bias by "the media?"
Sorry for all the scare quotes. I'm just trying to piece it together and am having no luck.
And it explains that these types of things "being fixed" is not really possible in a way that doesn't screw some group over ... you just kind of have to decide who you want to screw over.
Have a read it is very interesting.
[0] https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...
The other reply answered the rest of the question as I see it, so I will respond to this part. In your language, I was referring to the representative demographics of some social class or occupation or the like, though demographics fitting to historical ethical norms would apply here as well. And all of this is with respect to Western Democracies such as America, of course.
I should also mention here that when we weight these AI scales with the goal of maintaining arithmetic equality amongst demographics, everyone loses. In particular, justice dies for the many when this becomes the case, and it is indefensible to argue otherwise.