Unexpected sources of bias in artificial intelligence

I take race realism to be the sort of thing that posits an inherent relationship between skin color/other physical features, and other innate/intrinsic/genetic properties such as intelligence or criminality -- especially as a prior and when the trait is normative and negative. So "dark skinned people are less intelligent" is race realism, but "dark skinned people are treated poorly by the political establishment" is not race realism. (We're far afield, I don't mean for this thread to devolve into a definition or debate about race realism -- I'm just saying what I take race realism to mean.)

The important point isn't a precise characterization. Rather, the important point is this:

It's possible for race to be predictive without there being inherent differences between people of different physical attributes. Setup a system in which I take away all green people's life savings at the end of the day with probability .99 if they don't have a house and .5 if they do have a house. Both numbers are .2 for purple people. An unbiased lender can choose not to lend to green people without exhibiting statistical bias -- in fact, color has huge predictive power.

But that predictive power comes from the social system we setup specifically to screw green people, not from inherent differences that are causally related to color posited by race realism.

But this is an extremely obvious observation, so perhaps you meant something different by race realism? In any case, I don't think the disagreement you posited really exists.

The rest of the points you're making are covered elsewhere.

I deal with Real numbers every day, but I don't barge into discussions of epistemology and demand my axioms be used in discussions of a vaguely related but definitely different topic. And I'm careful in pop lectures to point out that Real numbers are just a mathematical object and no more more "real" than natural numbers.

If people in your statistics lectures don't know what bias means, then you should account for this in your teaching and lecturing. I know my statistics lecturer did.

I fail to see why you insist that the authors use a word in its statistical sense when they clearly aren't making a statistical argument -- especially since the the statistical sense of the word bias is ultimately derived from the common usage, and not the other way around. The use of 'bias' in the sense of 'prejudice' in English dates from the 1570s, well before the development of statistics [1].

If anything, we should complain that the statisticians overloaded a perfectly good English word, instead of coming up with some Greek or Latin neologism for their own technical use.

[1] http://www.etymonline.com/index.php?term=bias

> Similarly, I advocate against new-age uses of the word "energy" - energy is a specific conserved quantity,

Sounds like a stilted way to look at the world. If someone says 'I'm feeling really energetic today', would you object on the grounds that they haven't shown that their biochemical energy reserves are larger than normal, or that their body is burning calories at a faster rate than usual?

The overloading happened the other way around...

> What benefit do we gain when other people can't even understand the words we use?

Similarly, what benefit do we gain when we use a term of art in settings where most of the audience doesn't have the technical background to be quite sure what they're even discussing?

Also, there is a notion that's distinct from statistical bias and that is highly political and controversial. Today we can say "bias" and most everyone will know we're talking about that topic. Whatever word or phrase we use in place of "bias" will be just as muddy and ill-defined. And probably some mathematician will come along and build some incomplete model and name if after that new word. And then we'll have this conversation again. The problem you're trying to avoid is unavoidable.

The politics around racial discrimination and bias means that in reality, few people do the work to tease apart (1) and (2). In particular, I think many people are extremely uncomfortable with the consequences that (2) being true entail.

Hence 'racial bias' usually has a simpler meaning: something is biased if the decision conditioned on race is not identical to the unconditional decision. That encompasses both (1) and (2).

You seem to ascribe some sort of malicious motive to the authors, when what you really have a problem with is how the country talks about race and racial bias in general.

The point is that in general, statistically unbiased algorithms will not lead to racially unbiased (in the strict conditional outcome = unconditional outcome sense) results. You might think this is completely obvious as a practitioner in the field, but it is far from obvious to the general public. Hence why it's fair to point that out to the public.

> All I said is that we should name it something else

As I said elsewhere, some mathematician would come up with a concept, name it after that new thing, and then we're back here all over again because "bias" is inherently moral and political and has shades of meaning informed by moral and political opinion.

You won't have solved the precision problem, or even the overloaded terms problem.

> Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions.. I think this will mislead people

I agree "bias" is a politically powerful word.

I don't agree, however, that people fail to know and understand the difference between (1) and (2).

And I also think all people know that the term "bias" could mean both, and manage to communicate with one another using the full range of their vocabulary whenever it's unclear what's being discussed. See e.g. internet discussions around stop and frisk or WoD or racial profiling at airports.

People understand the difference between (1) and (2), even when bias is used to refer to both.

I might be wrong about that. A far more important prediction: I don't think there's anyone -- on either side of the debate -- whose position can be explained by their misunderstanding of the statistical meaning of "bias".

People understand the statistical meaning (or not), and call (2) bias regardless. Because they aren't referring to a mathematical object, they're referring to the effect something has within a larger social and political context. A context that cannot be perfectly modeled, and so is necessarily vague and difficult to pin down.

The fact that "bias" can have both meanings isn't a real problem. People are good communicators and, when they want, can easily distinguish between (1) and (2).

Saying that (2) is "not bias" is definitely a political statement when taking the common definition of bias. People are free to agree or disagree with that statement, but arguing via way of naming clash isn't a compelling argument (not saying you're making that argument, but GP is I think).

> Why do you want to conflate these two situations?

I don't think this is a real problem.