Hacker News Reader

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?

> I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form.

It's an example. The article mentions calculating recidivism rates to determine parole, which is a similar problem if you prefer that one.

> But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?

Most of the examples in the article have nothing to do with "bias" and are really the principal-agent problem. Misalignment of incentives.

Some website wants to maximize clicks rather than act in the user's interest, they create a machine to maximize clicks, the machine maximizes clicks. It's doing what they wanted it to do -- it's just not doing what you wanted it to do.

This category of problem has almost nothing to do with ML in particular. It's like identifying that charlatans often wear cotton and writing an article on the link between cotton and charlatans.

News algorithms tending to classifies stories as relevant reading based on relatively crude analysis of word frequencies and headline clickthroughs (as opposed to, say, whether the story added additional information or an alternative perspective, both of which would typically be negatively weighted by such an algorithm) is much more an ML problem than a principal-agent problem, unless one believes that "maximising clicks" is literally all anyone applying this technique aims to do, and that's the only reason why ML systems tend to optimise for simple goals like click-maximization rather than complex ones like breadth and salience of relevant information

There are obviously still unsolved problems in ML. But Facebook has billions of dollars. It isn't that they couldn't solve the problem if they wanted to. Yet they haven't.

> This category of problem has almost nothing to do with ML in particular.

The article is identifying a particular misconception that people might have, which is because a computer did it, the result will be (racially) unbiased.

It might be obvious to you that's not the case, since you're familiar with the ML techniques in question, but still valuable to a lay audience to make that point crystal clear.

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form.

Sorry, s/loan/allow out on bail/ or s/loan/show advertisement/ to go with examples in the article. Mathematically it's the same problem. I just default to thinking about loans since I've worked on loans and since most of the academic work on the problem is about lending.

But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality...

Thanks - you've perfectly illustrated my point as to why the term "bias" is misleading. The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

But unlike the loan example they are; that's precisely my point. In the loan example the algorithms (or actuaries' calculations) are often entirely accurate in identifying that for whatever reason $ISBLACK is associated with statistically significant increased likelihood of default after controlling for other variables, which may result in an unbiased estimation model assessing a particular person falls into an unacceptable default risk category (which FWIW isn't the same thing as accurately predicting a particular person won't pay, which is a more dangerous conflation of concepts...)

That doesn't apply to the examples discussed in the article - where Asians aren't blinking (or statistically more likely to be blinking), people trying to game a Twitter bot's corpus of conversations are not representative of human interaction patterns, nobody seriously believes social media bubbles represent reality and even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.

> even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.

That's actually the expected consequence of having different recidivism rates.

If some of the predictions of who will recidivate are wrong at random and a higher percentage of black people were predicted to recidivate then a higher percentage of black people will have been incorrectly predicted to recidivate.

If some of the predictions of who won't recidivate are wrong at random and a higher percentage of white people were predicted not to recidivate then a higher percentage of white people will have been incorrectly predicted not to recidivate.

In other words, X * error rate is more than Y * error rate because X is more than Y.

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

You can make a strong argument that, for example, the FB news feed algorithm leads to results that are unrepresentative of reality, because its input data (user clicks and likes) do not sufficiently represent reality accurately.

Or from the article: "Nikon’s confusion about Asian faces and HP’s skin tone issues in their face recognition software both seem to be the product of learning from skewed example sets." Nowhere here is the algorithm being blamed, but we have an example of results that at least some people find objectionable due to patterns in the input data.

Just to take a particular example, the algorithm's output has marked someone as blinking when they aren't, they're just Asian. That's a particular example of results being unrepresentative of reality.

As much as you want to talk about the algorithms in isolation, using ML also includes figuring out what input data to use and also how to properly interpret the output.

Replace the word 'algorithm' with 'person' or 'institution'.

The politics around racial discrimination and bias means that in reality, few people do the work to tease apart (1) and (2). In particular, I think many people are extremely uncomfortable with the consequences that (2) being true entail.

Hence 'racial bias' usually has a simpler meaning: something is biased if the decision conditioned on race is not identical to the unconditional decision. That encompasses both (1) and (2).

You seem to ascribe some sort of malicious motive to the authors, when what you really have a problem with is how the country talks about race and racial bias in general.

The point is that in general, statistically unbiased algorithms will not lead to racially unbiased (in the strict conditional outcome = unconditional outcome sense) results. You might think this is completely obvious as a practitioner in the field, but it is far from obvious to the general public. Hence why it's fair to point that out to the public.

(edit: Either confusion isn't a real problem, or else the confusion is inevitable and some new term will become the politically charged one with lots of contested meanings. In either case, the correct solution from your perspective should be more statistics education for more people rather than trying to change terminology.)

> All I said is that we should name it something else

As I said elsewhere, some mathematician would come up with a concept, name it after that new thing, and then we're back here all over again because "bias" is inherently moral and political and has shades of meaning informed by moral and political opinion.

You won't have solved the precision problem, or even the overloaded terms problem.

> Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions.. I think this will mislead people

I agree "bias" is a politically powerful word.

I don't agree, however, that people fail to know and understand the difference between (1) and (2).

And I also think all people know that the term "bias" could mean both, and manage to communicate with one another using the full range of their vocabulary whenever it's unclear what's being discussed. See e.g. internet discussions around stop and frisk or WoD or racial profiling at airports.

People understand the difference between (1) and (2), even when bias is used to refer to both.

I might be wrong about that. A far more important prediction: I don't think there's anyone -- on either side of the debate -- whose position can be explained by their misunderstanding of the statistical meaning of "bias".

People understand the statistical meaning (or not), and call (2) bias regardless. Because they aren't referring to a mathematical object, they're referring to the effect something has within a larger social and political context. A context that cannot be perfectly modeled, and so is necessarily vague and difficult to pin down.

The fact that "bias" can have both meanings isn't a real problem. People are good communicators and, when they want, can easily distinguish between (1) and (2).

Saying that (2) is "not bias" is definitely a political statement when taking the common definition of bias. People are free to agree or disagree with that statement, but arguing via way of naming clash isn't a compelling argument (not saying you're making that argument, but GP is I think).

> Why do you want to conflate these two situations?

I don't think this is a real problem.