Language contains human biases, and so will machines trained on language corpora

> Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving.

I don't believe that problem will ever be completely solvable. But I think the road to go is to make these assumptions always explicit. I.e. when the machine learning system derives a result, program it to additionally return a proof of how it came to this result. And also give a way to let the ML system return a list of all axioms and derivation rules that it has currently learned, so that they can independently be checked how much they are biased and can thus be corrected.

For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others.

This is not true.

Here's an oversimplified example. Suppose your machine learning system wants to predict something, e.g. loan repayment probabilities. One input might be a written evaluation by a loan officer.

When trained on a corpora of group X, the predicted probability might be:

    pred = a*written_evaluation + other_factors

(Using linear regression to make example simple.)

However, now lets suppose the written evaluation is biased to the tune of 25% against group Y. I.e., group Y has written scores that are 25% less than group X.

Then a new predictor which includes pairwise terms, trained on a corpora of group X and Y, will work out to be:

    pred = a*written_evaluation + 0.33*written_evaluation*isY + other_factors

This predictor would be unbiased. In general, if you have a biased input and the biasing factor is also present in your input, your model should correct the bias. (Obvious caveats: your model needs to be sufficiently expressive, etc.)

Interestingly, everyone's favorite bogeyman, namely redundant encoding ( http://deliprao.com/archives/129 ) will actually help fix this problem *even if you don't include the biasing factors in the model.

Which is to me rather a sign of how society changes meaning of the language to indoctrinate a kind of social propaganda. Greetings from George Orwell's 1984.

Yes, unfortunately the term is overloaded. "Bias" can also mean "making correct decisions that I wish were incorrect". That's why I explicitly defined "bias".

In the past era, e.g. 1980-2010, it was possible to use vague emotive language to support all kinds of disparate things. As a concrete example that I touch on in my post, and since racism is the loaded undercurrent of this example, we like to pretend that eliminating racial or sexual bias (in the sense of making wrong decisions) will get us proportional representation.

Algorithms are bringing us to an age where analytic philosophy is becoming really important. You can tell an algorithm to give you proportional representation, or you can tell it to be racially/sexually unbiased. But the algorithm will reflect reality and reality may not agree with your assumptions; you can't assume that asking for one will give you the other. So now we get into trolley problems: how much meritocracy/equal opportunity will you sacrifice to get proportional representation?

Unlike before, this is now a choice you need to explicitly and openly state and acknowledge.

Are there any overlaps or thoughts regarding Sapir Whorf here in your research?

I'm not convinced by your skepticism about correcting prejudiced bias. Debiasing certainly gives the AI a understanding of the world than the original (biased) language dataset, but it's not necessarily less complete - or less accurate. After all, any one corpus is incomplete, and has biases based on the items that were chosen for it - which are likely to reflect the biases of the past, and of the person choosing the corpus. It may not be a "complete" or "accurate" reflection of today's world - let alone the future. So it's not at all clear to me that efforts to undo the bias will necessarily make it less "accurate".

I agree with the suggestion to de-bias the application and not the representation itself.

Recently I was using a version of Conceptnet Numberbatch (word embeddings built from ConceptNet, word2vec, and GloVe data that perform very well on evaluations) as an input to sentiment analysis. So its input happens to include a crawl of the Web (via GloVe) and things that came to mind as people played word games (via ConceptNet). All of this went into a straightforward support vector regression with AFINN as training data.

You can probably see where this is going. The resulting sentiment classification of words such as "Mexican", "Chinese", and "black" would make Donald Trump blush.

I think the current version is less extreme about it, but there is still an effect to be corrected: it ends up with slightly negative opinions about most words that describe groups of people, especially the more dissimilar they are from the American majority.

So my correction is to add words about groups of people to the training data for the sentiment analyzer, with a lot of weight, saying that their output has to be 0.

Interesting paper, thanks!

See how you sway the argument in your favour using words with negative connotations like "fairness through blindness" and "hurt meaning and accuracy". Nobody would want to deliberately blind or hurt something, would they? How about rebalance or recalibrate or re-correct.

A concrete analogy:

1) I have a meter measuring stick but I discover that it was made wrong, it is actually 2mm shorter than advertised. Every time I make a measurement with it I have to add 2mm to the measurement. Would it not be better to use a more accurate stick and not have to continually compensate?

> For example, if you're offering a search engine solution incorporating clustering, you'd be concerned that your search retrievals aren't associating racial minorities with pejorative prejudices.

If in the data racial minorities are associated with pejorative prejudices, this is plain distorting of the truth. If you are concerned about this associations, don't shoot the messenger, but the people writing the original texts.