Hacker News Reader

Comment by yummyfajitas | Hacker News Reader

I think we need to stop using the word "bias" and "prejudice". The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply incorrect decisions are being systematically made.

Only one example in this article is actually about incorrect decisions. Most examples are about unbiased learning systems doing exactly what they are designed to do, but people other than the designers wishing they did different things.

For example, consider "conflicting goals bias". This isn't a bias - the system is maximizing clicks exactly as desired. It's just that some random third party wishes the system were actually trying to mitigate a nonexistent psychological problem (namely stereotype threat) instead [1].

What they call "similarity bias" is the same thing. The system is attempting to show people stories they like (and probably does a good job at it), but the author wishes instead that the selection of stories was closer to what a journalist might choose.

Another social "bias" - namely "redlining"/redundant encoding/etc is actually the elimination of statistical bias. A lot of input metrics - e.g. SAT score, FICO score, etc - are biased in favor of non-Asian minorities (and against Asians) [2] and machine learning algorithms designed to find hidden patterns discover this bias and fix it.

Conflating "someone is doing X but I wish they did Y" with bias is not useful. It's also not useful to conflate the elimination of socially desirable biases with introducing new biases.

[1] Stereotype threat has repeatedly failed to replicate. Funnel plots suggest it only existed to begin with due to publication bias. http://www.sciencedirect.com.sci-hub.cc/science/article/pii/... https://dl.dropboxusercontent.com/u/85192141/2013-ganley.pdf https://en.wikipedia.org/wiki/Stereotype_threat#Failures_to_... https://replicationindex.wordpress.com/tag/stereotype-threat...

[2] See for example Figure 7 here: https://drive.google.com/file/d/0B-wQVEjH9yuhanpyQjUwQS1JOTQ...

View on HN

> The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply incorrect decisions are being systematically made.

Bias is not a word that was invented by statisticians[1], it's a word that statisticians adopted. It's just as legitimate to say that when a machine learning technique is trained with biased input that it produces biased output as it is to say that some particular machine learning method produces biased output from unbiased input. Either way, it's perfectly fine to say that the use of that technique results in bias even if that bias is in an aspect of the output that the algorithm wasn't selected to optimize.

It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias, and implying that what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.) Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.)

[1] http://etymonline.com/index.php?term=bias

The term "bias" and "prejudice" even outside statistics are defined by the dictionary as being about getting the wrong results: "preconceived opinion that is not based on reason or actual experience".

This agrees more or less with the statistical term, it's just less precise. That's also NOT what machine learning algorithms do.

If you want to declare an algorithm biased because it optimizes what it was designed to optimize rather than some random tangential goal, then the term has become meaningless. Similarly, my cell phone is broken because it can only make calls and is ineffective at pulling trains. A locomotive is also broken because it doesn't make tacos.

It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias...what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.)

I have no idea how you could possibly think I made this claim given that I explicitly listed two examples of algorithms which are biased. I then hinted at how algorithms can eliminate this bias. Consider rereading what I wrote.

If you want more details on how algorithms correct biased inputs (it's not based on "ether") read the blog post I linked to downthread: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.)

I have no idea why you think "race realist" agendas require purist libertarianism, or vice versa. That's a random political tangent and totally unrelated to this conversation. I'm beginning to think you are seeking to derail the conversation rather than discussing statistics in good faith.

In the hopes that I've misunderstood you, I will nevertheless provide you with a good faith clarification. I made no normative claims regarding libertarianism or anything else. The only normative claim I've made is that we should not wrongfully apply an existing term (bias) to completely unrelated concepts (having undesired social outcomes).

> That's also NOT what machine learning algorithms do.

A machine learning algorithm run on non-representative input could be said to lack experience with the portion of input that isn't represented in the training set.

E.g. I think it's pretty obvious that machine learning algorithms exhibit far more vernacular-bias than purely logical reasoning techniques.

> I made this claim given that I explicitly listed two examples of algorithms which are biased.

Your parent is pointing out the very real possibility that technically-unbiased algorithms can produce vernacularly-biased outputs. I.e., that statistic's definition of "bias" does not sufficiently capture the notion of bias as it's used in the vernacular.

I don't think you've refuted that claim.

At the end of the day, your approach toward side-stepping the issue of vernacular bias is intellectually lazy. Instead of tackling the problem head on, you're hiding behind a mathematical object that happens to have the same name as the actual thing under discussion.

> I'm beginning to think you are seeking to derail the conversation rather than discussing statistics in good faith.

I think your characterization of bias in terms of statistic's technical definition is already skimming the edges of good faith. You know what the reporter means when they say "bias", and that's not the meaning that statisticians use in technical settings.

I know what the reporter means: "bias" refers to "outcomes the reporter dislikes and thinks he can generate clickbait with". I'm explicitly advocating against this practice and in favor of using clear and precise language instead.

I'm not side stepping the issue. I'm advocating in favor of describing the problem with clear language - once that's done we can begin useful discussion on how to address it.

In precise terms, what do you think the problem is?

Didn't he make that clear?

> Your parent is pointing out the very real possibility that _technically-unbiased algorithms can produce vernacularly-biased outputs_. I.e., that statistic's definition of "bias" does not sufficiently capture the notion of bias as it's used in the vernacular.

> "bias" refers to "outcomes the reporter dislikes and thinks he can generate clickbait with"

That's quite a straw man for someone who claims others are trying to derail a conversation.

> I'm explicitly advocating against this practice and in favor of using clear and precise language instead.

The language you're suggesting masks the problem.

It's nice that statisticians came up with a mathematical object that happens to have a politically charged name, but the conflation you're making is not an argument, and doesn't address the underlying (inherently political) problem.

> In precise terms, what do you think the problem is?

The precise problem is that things like skin color etc. often have strong predictive power. We've recognized this unfortunate fact -- and its societal implications -- and decided to create laws and social norms that regulate (either restrict or sometimes even require) the use of these properties in certain high-impact decisions.

Unfortunately, algorithms sometimes use these features to make decisions in settings where a typical human wouldn't (or might even get into legal hot water if they did).

Notice that an algorithm can exhibit this behavior without being (statistically) biased.

Now, perhaps you think this problem isn't actually a problem. And you can certainly make the argument that any form of vernacular-discrimination should be OK as long as it doesn't exhibit statistical bias. But that's a separate conversation, and you can't defend that position by playing with definitions. And if you're not making that argument, then it should be clear why vernacular-bias differs from statistical-bias.

You and pessimizer have just demonstrated pretty clearly why Techcrunch's language choices are bad.

Pessimizer thinks the article is discussing inaccurate conclusions and that race realism is wrong ( https://news.ycombinator.com/item?id=13158797 ). You think race realism is accurate but we should ignore it. You both read the same article but drew totally different conclusions due to it's imprecise language.

Many examples in the article don't fit your description of the problem at all (e.g. difficulties in image processing, Tay the trolled chatbot, filter bubbles), so I don't think you are correctly describing what the article means by "bias" either. I can't figure it out either; the best I can come up with is "AI-related things that make the author have negative feelings".

I didn't argue in favor of or against your mysterious "vernacular-discrimination" - how could I when I don't even know what it is? The only thing that I argued is that the imprecise language used by Techcrunch is confusing people and that this is bad.

You seem to be missing a key point, which is that the definitions of racial bias are and have always been contentious and up for debate. The definitions have always been in flux and have evolved heavily over time. This is basically true for many politically contentious topics, as different parties repeatedly try to redefine and reframe terms to their advantage.

If you insist on precise language, then we basically can never have any conversation about race, racial bias, or any other topic where the terminology hasn't been completely nailed down.

The only thing I'm advocating for in this thread is clear thinking and clear communication - in particular I oppose overloading a perfectly well defined and clear statistical term.

Similarly, I advocate against new-age uses of the word "energy" - energy is a specific conserved quantity, not some mysterious unobservable thing that emanates from crystals.

What benefit do we gain when other people can't even understand the words we use?

It's almost as if you want to use a Motte-and-Bailey argument to confuse people ( http://slatestarcodex.com/2014/11/03/all-in-all-another-bric... ). Which is of course something that people do on this very topic: https://www.chrisstucchio.com/blog/2016/propublica_is_lying....

> in particular I oppose overloading a perfectly well defined and clear statistical term.

The overloading happened the other way around...

> What benefit do we gain when other people can't even understand the words we use?

Similarly, what benefit do we gain when we use a term of art in settings where most of the audience doesn't have the technical background to be quite sure what they're even discussing?

Also, there is a notion that's distinct from statistical bias and that is highly political and controversial. Today we can say "bias" and most everyone will know we're talking about that topic. Whatever word or phrase we use in place of "bias" will be just as muddy and ill-defined. And probably some mathematician will come along and build some incomplete model and name if after that new word. And then we'll have this conversation again. The problem you're trying to avoid is unavoidable.

It's 100% clear from the article that 'bias' is in the political, or common sense of the word. It's an existing, established usage, and not one that the authors simply made up.

I fail to see why you insist that the authors use a word in its statistical sense when they clearly aren't making a statistical argument -- especially since the the statistical sense of the word bias is ultimately derived from the common usage, and not the other way around. The use of 'bias' in the sense of 'prejudice' in English dates from the 1570s, well before the development of statistics [1].

If anything, we should complain that the statisticians overloaded a perfectly good English word, instead of coming up with some Greek or Latin neologism for their own technical use.

[1] http://www.etymonline.com/index.php?term=bias

> Similarly, I advocate against new-age uses of the word "energy" - energy is a specific conserved quantity,

Sounds like a stilted way to look at the world. If someone says 'I'm feeling really energetic today', would you object on the grounds that they haven't shown that their biochemical energy reserves are larger than normal, or that their body is burning calories at a faster rate than usual?

If you read your source, you'll realize the term "bias" referred to a slant in the floor of a ball game providing an advantage to one player. This slant would (I'm inferring) cause one player to have an advantage over another equally matched player depending on which side of the slant they are on.

This agrees perfectly with the statisticians definition, albeit applied to one particular example. Specifically, the slanted ball game field would provide a biased measurement (in the statistical sense) of player ability. Score 1 for the mathematicians.

I agree with you that this techcrunch reporter is using the term in some other mysterious "vernacular" way. It's this usage that I'm objecting to since it's misleading and confusing.

If someone says 'I'm feeling really energetic today', would you object on the grounds that they haven't shown that their biochemical energy reserves are larger than normal, or that their body is burning calories at a faster rate than usual?

If an article were discussing how coal is more "energetic" than nuclear energy, but was actually referring to energetic in the psychological sense (or maybe in the hippy energy crystal sense), I would in fact object. That would be similarly misleading.

> This agrees perfectly with the statisticians definition, albeit applied to one particular example. Specifically, the slanted ball game field would provide a biased measurement (in the statistical sense) of player ability. Score 1 for the mathematicians.

That's silly. It's not a competition between mathematicians and all other people for the 'correct', Platonic definition of a word -- a word is defined by how people use and interpret it. Logical, mathematical deduction from first principles is not the only way to look at the world.

Unless I'm mistaken, I think your objection can be summarized as 'we shouldn't talk about things that we don't have airtight definitions for (like racial bias or racial prejudice), because it might confuse people'.

I happen to strongly disagree, since grappling with fuzzy definitions is precisely how we get to having better definitions. Otherwise, we should give up on ever talking about anything except math, where we have clear axioms to work with.

Even in math, people were profitably doing calculus before we had clear definitions for infinitesimals, integrals, derivatives, or proofs that our procedures produced correct results. The logical machinery and deductive logic often comes after people spent time grappling with fuzzy, intuitive things with unclear definitions.

No. I give a concrete example of what I want in this comment: https://news.ycombinator.com/item?id=13161470

Specifically, the word "bias" makes virtually everyone think of one particular concept. I claim expanding the word "bias" to describe two separate concepts misleads people into thinking the second concept is the same as the first.

My sole objection is with confusing people by conflating multiple disparate concepts under one label, which is already a perfectly good label that clearly refers to something else (according to everything besides throwaway's mysterious and uncited "vernacular definition").

> You think race realism is accurate but we should ignore

No, I do not think race realism is accurate. I just think that race realism is not an adequate defense of racial discrimination, AND ALSO that race realism is inaccurate.

Pessimizer and I agree.

> Many examples in the article don't fit your description of the problem at all

You asked me what I think the problem is, not what I think the article's author thinks the problem is. The fact that I emphasize different things is not proof that I misunderstand the article's author.

> The only thing that I argued is that the imprecise language used by Techcrunch is confusing people and that this is bad.

Who, exactly, is supposed to be confused?

Subject matter experts should know the difference, and if they don't, that's really their own problem.

The public at large doesn't even know what statistical bias is, so why in the world would they be confused?

No matter how confusing the current state of affairs is, it would be far more confusing if suddenly everyone starting using bias to mean statistical bias. Mostly because 99.9% of the people using the term wouldn't have an adequate grasp of mathematics to even know to which object they are referring. So if your priority is clarity, then keeping around the messy moralized and politicized vernacular notion of bias is far preferable.

> I didn't argue in favor of or against your mysterious "vernacular-discrimination" - how could I when I don't even know what it is?

Not everything in the world can be mathematically formalized. Most things don't even have fixed meaning over time. If your criticism is genuinely about imprecision, you've got a lot of much lower hanging fruit than "discrimination" and "bias"...

The precise problem is that things like skin color etc. often have strong predictive power...Notice that an algorithm can exhibit this behavior without being (statistically) biased. - throwaway729, 2 hours ago https://news.ycombinator.com/item?id=13159539

No, I do not think race realism is accurate. - throwaway729, 22 minutes ago https://news.ycombinator.com/item?id=13160709

Whoops!

The fact that I emphasize different things is not proof that I misunderstand the article's author.

Fine - what do you believe the author thinks "bias" means? I.e. what is the underlying concept behind all his examples?

Who, exactly, is supposed to be confused?

Pessimizer, for one, who thinks the problem is inaccurate predictions. 2 hours ago you disagreed with this, but 22 minutes ago you agreed. I'm getting kind of confused.

If your criticism is genuinely about imprecision, you've got a lot of much lower hanging fruit than "discrimination" and "bias"...

Sure, but this is my field. I give talks about misuse of p-values and the multiple comparison problem, even though there might be other issues worth worrying about. It's just what interests me. The fact that bigger problems exist doesn't mean we shouldn't fix this one.

> Whoops!

I take race realism to be the sort of thing that posits an inherent relationship between skin color/other physical features, and other innate/intrinsic/genetic properties such as intelligence or criminality -- especially as a prior and when the trait is normative and negative. So "dark skinned people are less intelligent" is race realism, but "dark skinned people are treated poorly by the political establishment" is not race realism. (We're far afield, I don't mean for this thread to devolve into a definition or debate about race realism -- I'm just saying what I take race realism to mean.)

The important point isn't a precise characterization. Rather, the important point is this:

It's possible for race to be predictive without there being inherent differences between people of different physical attributes. Setup a system in which I take away all green people's life savings at the end of the day with probability .99 if they don't have a house and .5 if they do have a house. Both numbers are .2 for purple people. An unbiased lender can choose not to lend to green people without exhibiting statistical bias -- in fact, color has huge predictive power.

But that predictive power comes from the social system we setup specifically to screw green people, not from inherent differences that are causally related to color posited by race realism.

But this is an extremely obvious observation, so perhaps you meant something different by race realism? In any case, I don't think the disagreement you posited really exists.

The rest of the points you're making are covered elsewhere.

I deal with Real numbers every day, but I don't barge into discussions of epistemology and demand my axioms be used in discussions of a vaguely related but definitely different topic. And I'm careful in pop lectures to point out that Real numbers are just a mathematical object and no more more "real" than natural numbers.

If people in your statistics lectures don't know what bias means, then you should account for this in your teaching and lecturing. I know my statistics lecturer did.

When there exist two sets of meanings for a word -- one vernacular, the other technical -- we shouldn't try to insist that the technical meaning over-ride the vernacular meaning. That's almost always a losing battle.

Insisting that a system isn't biased in the vernacular sense due to a particular technical notion of bias completely misses the point. And we almost never have enough understanding about the world to make a definitive case that the system isn't biased in the vernacular sense.

My claim is, however, that when discussing statistics we should stick to the statistics terms. If we want to discuss a new concept then we should make up a new word for it.

To do otherwise risks having people get confused as to what underlying concepts are actually being discussed. For instance, go read the HN comments here: https://news.ycombinator.com/item?id=13004790 Why risk such confusion when we can just use two different words for two different concepts?

However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed:

Bias: prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair.

Prejudice: preconceived opinion that is not based on reason or actual experience.

This are just informal versions of the statistical term.

To translate this to another topic, if we are discussing global warming I don't think we should use the term "warm" to reference "feelings in my heart". Why would we want to confuse whether we are discussing the temperature of the earth or my feelings?

If this were a conversation about a statistics paper or textbook, or a particular ML algorithm, you might have a point.

But we're not discussing statistics. We're discussing the social impact of ML algorithms.

> Why risk such confusion when we can just use two different words for two different concepts?

The problem with this is that we'll always have that confusion because most people aren't statistics experts.

So it's up to the domain experts to know when one meaning or the other is intended. That's true in every domain.

> However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed

Dictionary definitions don't fully capture the vernacular meaning of words -- especially politically charged words such as "bias".

> This are just informal versions of the statistical term.

But when real people use the term bias in real conversations, they aren't typically approximating the statistical meaning. Rather, they are typically referring to some a priori ethical limitations on decision making.

For example, most people would say that taking skin color into account when approving a housing loan is a form of bias. But they don't mean that in any particularly technical way. There absolutely is moral and political content in the term "bias".

For example, when equal lending laws were passed, an unbiased algorithm definitely would've considered race when choosing the outcome of a loan application. African Americans were systematically discriminated against in employment decisions, and often the first to be let go when hard times hit. So a completely unbiased loan officer could come to the statistically valid conclusion that people of color were far more likely to default. None-the-less, a vase majority of people today would agree that the loan officer was exhibiting racial bias. And they're not coming to that conclusion due to innumeracy or some vacuous personal preference. They're perceiving a vast network of causal relationships between decisions that reinforce a systematic bias, even when many of those decisions aren't locally statistically biased

> I don't think we should use the term "warm" to reference "feelings in my heart".

And I agree, but I agree because non-experts know and understand the difference. And because the term "global warming" with the alternative meaning isn't already in wide-spread vernacular use.

> But when real people use the term bias in real conversations, they aren't typically approximating the statistical meaning. Rather, they are typically referring to some a priori ethical limitations on decision making.

What many common people mean by bias is approximately the statistical meaning. For example, if the algorithm accurately predicts the number of white defendants who fail to appear in court but overestimates the number of black defendants who fail to appear, people in common usage will call that bias. Which it is, in the statistical sense.

But if a higher percentage of black defendants actually fail to appear than white defendants, and the algorithm accurately predicts that outcome, that isn't bias in the statistical sense. And the people who try to conflate those things are doing this:

http://slatestarcodex.com/2014/11/03/all-in-all-another-bric...

The second thing is not only not bias, it's not morally objectionable. Statistical bias is the thing which is morally objectionable. Calling something else "bias" is not just inaccurate, it leads directly to a false moral conclusion.

> What many common people mean by bias is approximately the statistical meaning.

And many people don't. The difference is that I'm not asserting I have the one true definition, but rather recognizing that "bias" is a term with inherent political and moral content. And that pretending that this isn't (or shouldn't be) the case just because statisticians use the term in a technical setting is extraordinarily intellectually lazy.

Most people are not statisticians. No serious mathematician has ever asserted that the axioms of Real numbers settle all epistemological debates. Like-wise, I see no reason why a statistical term of art should be used to clear up the various denotations and connotations surrounding "bias" and "discrimination".

> that isn't bias in the statistical sense.

That depends on what the goal of the algorithm is.

If the goal is to accurately predict whether people will appear, then it's working.

But if the goal is to accurately predict when people will choose not to show up, then there's certainly a chance that it's not working. For example, in the case where there's some significant correlation between race and access to transportation infrastructure. (Which isn't so much the case today, but certainly was 60 or so years ago.)

The distinction between choice and circumstance isn't immaterial -- the latter is not a flight risk even if they gum up the legal system via delays. So if the judge is interpreting "90% won't show" as "90% flight risk" in the above hypothetical world, then there's a huge problem. You could argue that this is the judge's problem, and I won't necessarily disagree. But UI bugs are still bugs, and this particular form of UI bug is pretty specific to ML systems.

> And the people who try to conflate those things are doing this

Or, perhaps they're far better at mathematical modeling of complex systems than you give them credit for; e.g., by seeing ways in which locally statistically unbiased decisions can magnify the impact of bias that exists in other parts of a system.

We will never have a complete mathematical model of the world and the history of human civilization, so I don't see any reason why we should insist that all bias needs to be what a statistician calls bias in his workplace.

> The second thing is not only not bias, it's not morally objectionable

This is a separate discussion I don't want to have in this thread, because we're already barely on-topic.

But suffice it to say that conflating technical and non-technical definitions is not a reasonable defense of this position.

> And many people don't. The difference is that I'm not asserting I have the one true definition, but rather recognizing that "bias" is a term with inherent political and moral content. And that pretending that this isn't (or shouldn't be) the case just because statisticians use the term in a technical setting is extraordinarily intellectually lazy.

Your assertion seems to be that not using the same word to describe separate and distinguishable things in a context where the distinction matters is intellectually lazy.

> If the goal is to accurately predict whether people will appear, then it's working.

> But if the goal is to accurately predict when people will choose not to show up, then there's certainly a chance that it's not working. For example, in the case where there's some significant correlation between race and access to transportation infrastructure. (Which isn't so much the case today, but certainly was 60 or so years ago.)

The goal is to predict whether people will appear. The bail-bondsman has to know whether to issue the bail-bond.

If your issue is transportation infrastructure then the only way to solve it is by fixing the transportation infrastructure, not by bankrupting the bail-bondsman.

> This is a separate discussion I don't want to have.

> But suffice it to say that conflating technical and non-technical definitions is not a reasonable defense of this position.

That's not the point -- the problem is that conflating technical and non-technical definitions is a way to avoid having that discussion.

Effectively everyone agrees that statistical bias is wrong. If you're on the wrong end of it you're being treated unfairly. If you can convince someone that something is this "bias" when everyone agrees bias is wrong then you've convinced them that the thing is wrong.

But if all you've done is broaden the scope of what bias means to encompass something that not everyone agrees is wrong then you've accomplished nothing of virtue. And more than likely misled some number of people who haven't managed to piece together that the "bias" they understand to be wrong and the "bias" you understand to be present are non-overlapping.

> That's not the point -- the problem is that conflating technical and non-technical definitions is a way to avoid having that discussion.

On this we can 100% agree.

Choosing "bias" to mean "statistical bias", and then actively ignoring anything else that is currently called bias, is an extremely political choice.

Are you seriously asserting that naming clashes validate political opinions, or that some large group of people's political opinions would suddenly change if only they knew the statistical meaning of bias?

No one said we should ignore this other concept. All I said is that we should name it something else.

Yes, I think this use of "bias" is meant to mislead. Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions (that's what "bias" means in the dictionary and in most common uses of the term).

There are two separate concepts:

1) An algorithm refuses to issue loans to blacks even though those blacks would pay the loans back.

2) An algorithm refuses to issue loans to blacks because they would not pay the loans back.

You want to use the term "bias" to describe both situations; I think this will mislead people. Specifically most people will think "bias" means we live in situation (1). I want to use "bias" to refer to (1) and to use a separate term to refer to (2).

Why do you want to conflate these two situations?

Replace the word 'algorithm' with 'person' or 'institution'.

The politics around racial discrimination and bias means that in reality, few people do the work to tease apart (1) and (2). In particular, I think many people are extremely uncomfortable with the consequences that (2) being true entail.

Hence 'racial bias' usually has a simpler meaning: something is biased if the decision conditioned on race is not identical to the unconditional decision. That encompasses both (1) and (2).

You seem to ascribe some sort of malicious motive to the authors, when what you really have a problem with is how the country talks about race and racial bias in general.

The point is that in general, statistically unbiased algorithms will not lead to racially unbiased (in the strict conditional outcome = unconditional outcome sense) results. You might think this is completely obvious as a practitioner in the field, but it is far from obvious to the general public. Hence why it's fair to point that out to the public.

(edit: Either confusion isn't a real problem, or else the confusion is inevitable and some new term will become the politically charged one with lots of contested meanings. In either case, the correct solution from your perspective should be more statistics education for more people rather than trying to change terminology.)

> All I said is that we should name it something else

As I said elsewhere, some mathematician would come up with a concept, name it after that new thing, and then we're back here all over again because "bias" is inherently moral and political and has shades of meaning informed by moral and political opinion.

You won't have solved the precision problem, or even the overloaded terms problem.

> Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions.. I think this will mislead people

I agree "bias" is a politically powerful word.

I don't agree, however, that people fail to know and understand the difference between (1) and (2).

And I also think all people know that the term "bias" could mean both, and manage to communicate with one another using the full range of their vocabulary whenever it's unclear what's being discussed. See e.g. internet discussions around stop and frisk or WoD or racial profiling at airports.

People understand the difference between (1) and (2), even when bias is used to refer to both.

I might be wrong about that. A far more important prediction: I don't think there's anyone -- on either side of the debate -- whose position can be explained by their misunderstanding of the statistical meaning of "bias".

People understand the statistical meaning (or not), and call (2) bias regardless. Because they aren't referring to a mathematical object, they're referring to the effect something has within a larger social and political context. A context that cannot be perfectly modeled, and so is necessarily vague and difficult to pin down.

The fact that "bias" can have both meanings isn't a real problem. People are good communicators and, when they want, can easily distinguish between (1) and (2).

Saying that (2) is "not bias" is definitely a political statement when taking the common definition of bias. People are free to agree or disagree with that statement, but arguing via way of naming clash isn't a compelling argument (not saying you're making that argument, but GP is I think).

> Why do you want to conflate these two situations?

I don't think this is a real problem.

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?

> I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form.

It's an example. The article mentions calculating recidivism rates to determine parole, which is a similar problem if you prefer that one.

> But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?

Most of the examples in the article have nothing to do with "bias" and are really the principal-agent problem. Misalignment of incentives.

Some website wants to maximize clicks rather than act in the user's interest, they create a machine to maximize clicks, the machine maximizes clicks. It's doing what they wanted it to do -- it's just not doing what you wanted it to do.

This category of problem has almost nothing to do with ML in particular. It's like identifying that charlatans often wear cotton and writing an article on the link between cotton and charlatans.

> This category of problem has almost nothing to do with ML in particular.

The article is identifying a particular misconception that people might have, which is because a computer did it, the result will be (racially) unbiased.

It might be obvious to you that's not the case, since you're familiar with the ML techniques in question, but still valuable to a lay audience to make that point crystal clear.

News algorithms tending to classifies stories as relevant reading based on relatively crude analysis of word frequencies and headline clickthroughs (as opposed to, say, whether the story added additional information or an alternative perspective, both of which would typically be negatively weighted by such an algorithm) is much more an ML problem than a principal-agent problem, unless one believes that "maximising clicks" is literally all anyone applying this technique aims to do, and that's the only reason why ML systems tend to optimise for simple goals like click-maximization rather than complex ones like breadth and salience of relevant information

There are obviously still unsolved problems in ML. But Facebook has billions of dollars. It isn't that they couldn't solve the problem if they wanted to. Yet they haven't.

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form.

Sorry, s/loan/allow out on bail/ or s/loan/show advertisement/ to go with examples in the article. Mathematically it's the same problem. I just default to thinking about loans since I've worked on loans and since most of the academic work on the problem is about lending.

But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality...

Thanks - you've perfectly illustrated my point as to why the term "bias" is misleading. The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

You can make a strong argument that, for example, the FB news feed algorithm leads to results that are unrepresentative of reality, because its input data (user clicks and likes) do not sufficiently represent reality accurately.

Or from the article: "Nikon’s confusion about Asian faces and HP’s skin tone issues in their face recognition software both seem to be the product of learning from skewed example sets." Nowhere here is the algorithm being blamed, but we have an example of results that at least some people find objectionable due to patterns in the input data.

Just to take a particular example, the algorithm's output has marked someone as blinking when they aren't, they're just Asian. That's a particular example of results being unrepresentative of reality.

As much as you want to talk about the algorithms in isolation, using ML also includes figuring out what input data to use and also how to properly interpret the output.

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

But unlike the loan example they are; that's precisely my point. In the loan example the algorithms (or actuaries' calculations) are often entirely accurate in identifying that for whatever reason $ISBLACK is associated with statistically significant increased likelihood of default after controlling for other variables, which may result in an unbiased estimation model assessing a particular person falls into an unacceptable default risk category (which FWIW isn't the same thing as accurately predicting a particular person won't pay, which is a more dangerous conflation of concepts...)

That doesn't apply to the examples discussed in the article - where Asians aren't blinking (or statistically more likely to be blinking), people trying to game a Twitter bot's corpus of conversations are not representative of human interaction patterns, nobody seriously believes social media bubbles represent reality and even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.

> even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.

That's actually the expected consequence of having different recidivism rates.

If some of the predictions of who will recidivate are wrong at random and a higher percentage of black people were predicted to recidivate then a higher percentage of black people will have been incorrectly predicted to recidivate.

If some of the predictions of who won't recidivate are wrong at random and a higher percentage of white people were predicted not to recidivate then a higher percentage of white people will have been incorrectly predicted not to recidivate.

In other words, X * error rate is more than Y * error rate because X is more than Y.

Given the findings on stereotype accuracy are some of the best-replicated in psychology, I've always found stereotype threat a humorous idea. It implies stereotypes are accurate because they trick us into perpetuating them. Can you think of a more absurd conspiracy theory? It's in keeping with the very popular notion that you can change reality with words, and thus is commonly said to be true in hopes of making it so.

Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias".

But about machine learning... a system can learn from a set of outcomes which are the result of a "biased" system, that is to say tainted with an incorrect Bayesian prior which is not properly corrected, such as let's say courts in a racist society or whatever. It would learn the same bias. Because its goal is to maximize compatibility with the outcomes that the humans did. So it perpetuates those weights.

The problem is that we don't know whether the human decisions matched the reality. "Did the person commit the crime" for example. We might have to wait until more unbiased estimators for such activity come along, and throw away old historical data.

It's sort of like when Black-Scholes became a self-fulfilling prophecy for valuing derivatives, but only after it became widespread. The market started using Black-Scholes to value derivatives, so it became the best model to predict the value of derivatives. But until then, other models might have fit the historical data better.

Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias".

If you interpret the word "bias" statistically - rather than the way innumerate reporters trying to sound profound use it - then it's pretty straightforward to do so.

"Knowledge" represents a fixed snapshot of your information about the world.

"Bias" represents a tendency for your knowledge to systematically differ from reality even as more data is gathered.

In programming terms, knowledge is the data in your DB at a fixe dtime. Bias is a tendency for my inserts to fail while yours succeed.

If the goal of a system is to predict human behaviors, then it will in fact do that. That's not "bias", that's just building a system with the goal of matching human behavior and getting what you asked for.

However, if the inputs to your algorithm are biased predictors of the output your outputs can still be unbiased. The tendency of machine learning systems is to detect hidden patterns in the data; biased inputs are just another pattern.

I wrote a blog post earlier this year explaining this in detail: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

So you know, I've gone ahead and submitted that: https://news.ycombinator.com/item?id=13158883

I do think it's important for people to be more clear about the goals they have for these systems. Do you want them to be accurate regardless of anything else, or do you want them to accomplish particular social goals by deliberately bending the results? There's nothing necessarily wrong with the latter, though you will inevitably find once you're being clear about the latter that even people who are generally ideologically aligned with each other will discover they have different and mutually conflicting goals...

Anyhow, either way, until you have a clear specification about goals you can't determine whether you're accomplishing them. A vague goal of being "nondiscriminatory" doesn't cut it for computers... "nondiscriminatory" is a set of possibilities, not a unique specification. (And if you disagree about my claim it's a set, just try to sit down with one of the aforementioned ideologically-aligned people and try to hammer it out to a coding level of detail.)

Not only that, there are impossibility theorems that say you can't meet everyone's definition of "nondiscriminatory" except in a few trivial cases.

https://arxiv.org/pdf/1609.05807v1.pdf

I.e. every algorithm will be "discriminatory" unless you predictor is perfect (i.e. gets 100% of decisions right). It's just a question of which kind of discriminatory they are. The same is true of any human decision process.

Unfortunately this means that we will be subjected to a slew of innumerate reporters posting "XXX is discriminatory" articles, regardless of how decisions are made.

Say I have two categories, programming articles and non- programming articles, and some other data about each article. And I want to predict whether the article will be interesting or not. And I want to be fair to interesting non-programming articles by having the same proportion of false negatives to correct positives in the non-programming subset of articles as in the programming subset of articles.

Is there a technical term for that in statistics?

It's like trying to get a representative sample, but only representative in one specific way (topic), and deliberately non representative in another (interestingness)

I think this could get at one of the things people mean, and it might be interesting to see how this trades off against overall accuracy or representativeness in other categories.

Reply on news.ycombinator.com