Coauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we don't say, and what the implications are. Happy to engage in informed discussion :)
Near as I can tell, your paper shows that these "biases" result in significantly more accurate predictions. For example, Fig 1 shows that a machine trained on human language can accurately predict the % female of many professions. Fig 2 shows the machine can accurately predict the gender of humans.
Normally I'd expect a "bias" to result in wrong predictions - but in this case (due to an unusual redefinition of "bias") the exact opposite seems to occur.
(Drawing on your analogy with stereotypes, it's probably also worth linking to a pointer on stereotype accuracy: http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... http://spsp.org/blog/stereotype-accuracy-response )
"In AI and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action (4). Yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior. Here, we will call such biases “stereotyped” and actions taken on their basis “prejudiced.”"
This definition is not unusual. This is about inferences that are wrong in the sense of prejudiced, not necessarily inaccurate.
In any case, the paper suggests that this "bias" or "prejudice" is better described as "truths I don't like". I'm asking if the author knows of any cases where they are actually not truthful. The paper does not suggest any, but maybe there are some?
Think for example of an inductive bias. If I see a couple of white swans, I may conclude that all swans are white, and we all know this is wrong. Similarly, I may conclude the sun rises everyday, and for all practical purposes this is correct. This kind of bias is neither wrong nor right, but, in the words of the article "a necessary prerequisite for intelligent action", because no induction/generalization would be possible without it.
There are undoubtedly examples where the prejudiced kind of biases lead to both truthful and untruthful predictions, but that seems beside the point, which is to design a system with the biases you want, and without the ones you don't.
From what I understand, the fear surrounding embedding human stereotypes into ML systems is that the stereotypes will get reinforced. In some way or form, there will be less equality of opportunity in the future than exists today, because machines will make decisions that humans are currently making. Societal norms evolve over time, yet code can become locked in place.
Is your takeaway from this paper that we, as the creators of intelligent machines, should allow them to continue to making "positively" right assumptions simply because that's the way we, as humans, have always done them? Is "positively" right, in your opinion, in all cases equivalent to "normatively" right?
Given how accurate human corpora is at predicting things like gender distribution in jobs for instance, wouldn't making an "unbiased" corpora make an inaccurate AI?
Shouldn't we be careful in implying things like the biases and solutions to said biases? For instance, I'd like to know if my algorithm for filtering job applicants is trying to undo the injustices of the world in addition to finding the best candidates.
- An AI correctly infers (simply by reading text) that a physicist is male and a nurse is female.
- An AI correctly infers the gender of humans with androgyonous names.
- An AI infers insects are unpleasant and flowers are pleasant to humans.
- An AI also infers that African American names are more likely to be associated with unpleasantness than European names.
[edit: to those who dislike this comment, can you tell me what you object to? Which of my concrete examples is not in the paper?]
best probably to read that one first.
Unless the link was changed in the few minutes since you posted your comment, the link for the article is the original Science paper (http://science.sciencemag.org/content/356/6334/183.full)
Page with actual link:
http://science.sciencemag.org/content/356/6334/183/tab-pdf
Link to PDF itself:
http://science.sciencemag.org/content/sci/356/6334/183.full....
If I look at glove & WordNet usage e.g. for topic extraction, bagging / clustering or semantic similarity would you say we would need to get rid of such a bias, e.g. create something like a Geiger counter for NLP.
Alternative view - when doing sentiment analysis / classification would you say that such a bias actually helps to identify a type of sentiment in a doc / sentence.