Cognitive roadblocks to reconciling merit and diversity | Hacker News Reader

Cognitive roadblocks to reconciling merit and diversity

hbr.org

33 pointswallflower34 comments

I used to be really concerned and generally opposed to such biases. But after careful thought, I'm now very uncertain.

I really want to know how much of this "bias" is actually just an informative and useful prior.

As an example, suppose you have two candidates from Princeton. One is black, the other is asian. With high probability, the asian candidate is smarter - he had to pass through a much more rigorous college admissions filter than the black person.

https://news.ycombinator.com/item?id=11904256

Now given equal evaluations, it probably makes sense to hire the Asian person over the black person. (Similar reasoning suggests you should prefer an Asian to a white person.) The reason is simply that base rates matter. Bayes rule says that P(good candidate|evidence) = P(evidence|good candidate) P(good candidate)/P(evidence).

The term P(good candidate) is known as the base rate - as simple arithmetic demonstrates it's important information.

So assuming the base rate for various groups differs, these "biases" might actually not be biases at all - they might be informative. If we eliminate these biases from our process, we might actually make worse decisions [1]. Unfortunately this article doesn't address that question at all.

See also this body of research, which suggests that many racial/sexual stereotypes are actually useful priors and improve human decisionmaking: http://www.rci.rutgers.edu/~jussim/unbearable%20accuracy%20o...

I'm also curious to hear arguments about the ethics of using such information in decision making, assuming they are in fact informative.

[1] For those unfamiliar with statistics, I've got a simple article here illustrating the arithmetic of base rates. Clickbait title: A gay person donates blood which tests negative for HIV. A straight person donates untested blood. Which is safer? The answer will SHOCK you. https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate...

"Biases" are a species of uninformed preference. It is really unlikely that satisfying your biases is going to lead to better decision-making. This goes for just about any mismeasure and folk wisdom, as Google has found out.

Multiple dimension of Google's hiring process have been revised as they found they were getting sub-optimal results from the usual kind of "hiring people like me" approach.

For example, it used to be a good bet that men made better orchestra musicians. Until they made auditioners take off their shoes and play behind a screen. You can't detect that bias without trying very hard to wash it out. If men were substantially ALL of the successful orchestral musicians, it would of course look like a completely justified "bias."

First of all, base rates matter. Here's a simplified proof:

Suppose you want to know if X is true. You have evidence E which is correlated with X - suppose A = P(E|X) > 0.5 and B = P(!E|!X) > 0.5. Bayes rule shows:

P(X|E) = P(E|X) P(X) / P(E) = P(E|X)P(X) / (P(X)A + (1-B)P(!X)) = P(X)A / (P(X)(A+1-B) + 1-B)

Differentiating this w.r.t. P(X) shows that it monotonically increases with P(X). QED.

Now I'm not disputing - at all - that human biases exist. Far from it! What I'm asking is whether the behaviors discovered in this article are biases (errors that result in worse decisions) or merely accurate use of priors (using correct base rate knowledge to make better decisions).

In the case of orchestras the answer is pretty clearly yes. Similarly in the case of baseball. In college admissions and lending the answer is likely to be no [1]. This article completely ignores the question!

[1] https://randomcriticalanalysis.wordpress.com/2015/05/16/on-c... http://ftp.iza.org/dp8733.pdf https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t...

1. Argue that yummyfajitas is known to often be wrong, and in fact terribly wrong, thus justifying bias against acceptance of any views advanced by yummyfajitas.

2. Spread this now rationally-supported (on base rate grounds) bias among many people.

3. Find some time later that yummyfajitas' views are overwhelmingly rejected by persons reached in step 2.

4. Declare bias established in step 1 to be retroactively objectively correct after observing how yummyfajitas' posts fare with observers, and note that even yummyfajitas will not be able to argue with it, as the correctness will be demonstrated by yummyfajitas' own argument in comment above.

The chain of reasoning you advocate for is circular - the validity of step 1 is dependent on the conclusion in step 4, but step 4 depends on step 1.

The same is not true of the chain of reasoning I suggested.

It's amusing to read this thread right next to the "The Artist Leaving the Googleplex" one.

Nice explanation. I agree with your point. It's not natural for most people to perform statistical judgements. It's not obvious that base rates matter even in the presence of evidence. It's true as well that we should not forget that intuitive impressions of evidence are often exaggerated.

However, we would do well in not forgetting other biases like the Halo Effect that may at play during any interview process.

To get at the ethics of it, even if what you say is true, one must understand the Matthew Effect and self-fulfilling prophesy. These are just as much a force of nature as natural selection and the invisible hand.

Einstein called compounding growth one of the most powerful forces in the universe. The Matthew Effect is compounding inequality over generations.

Inequality isn't compounding though (at least this century). It's shrinking rapidly.

http://www.maxroser.com/economic-world-history-in-one-chart/ https://twitter.com/MaxCRoser/status/745602914123603968

But I'm curious though - by what mechanism will such decisions actually cause the Matthew effect? For examples in that article, it's somewhat clear - e.g., the famous mathematician becomes more famous because people notice his papers. But now, in employment, suppose I will make better decisions by favoring an equally qualified Tamil over a Bihari. How will that cause Tamil-Bihari inequality to increase at an exponential rate?

Also, even if some groups do get a higher growth rate than others, why is that relevant? Concretely, what ethical or moral reason do we have for choosing these arbitrary groups to care about?

I can't make head or tails of this chart. How many people had a salary between 500$ and 1000$ in the blue curve? The area under the curve is completely distorted by the logarithmic x-axis. Also, doesn't the second graph on https://ourworldindata.org/global-economic-inequality suggest that inequality has been steadily rising?

The second graph suggests inequality has been decreasing during my lifetime. (I was born after 1980.)

On your first point: Inequality among nations is in decline, as you note.

Inequality among individuals within nations, however, is on the rise. [1] I think this is as important as inequality among nations.

[1] https://en.wikipedia.org/wiki/Distribution_of_wealth

No, inequality among individuals across the world is in decline. That's what my graph shows.

Why is inequality within arbitrary borders as important? There are a near infinite set of possible borders - what makes your choice of borders morally meaningful?

Inequality within arbitrary borders is important because economic environments, and therefore effects on individuals, are still defined by those arbitrary borders.

AFAIK, it matters very little to my daily life or even to my two decade outlook what happens to wealth inequality within, say, Rwanda. But it matters very much that increasing wealth inequality (and consequent pricing phenomena) in the Bay area is driving away critical infrastructure operators.

People upon whom my community depends: teachers, firefighters, police, sewage treatment operators, water company employees, ambulance drivers, road maintenance workers - the people who provide the necessities that sustain my community - are being driven away by increasing wealth inequality. They're priced out of living within (now) about 2.5 hours from where they work. In my value system, this is not good. Not for them, for me, or for my neighbors (some of whom must leave due to property tax increases).

Global statistics aren't useful in understanding local effects. Each community, whatever the particular definition of community may be, is at the mercy of what happens to it and those communities upon which it depends.

I get that globally, humanity is making real progress in quality of life. And I'm very glad for that.

But like the future, I see that it's not evenly distributed. We should work on that.

The issue you are discussing is the Malthusian scarcity of housing caused by exclusionary zoning. Specifically, SF has N houses but K > N people who want to live there. Arithmetic implies K-N people need to live further out.

Reducing income inequality cannot fix this. Only building more houses can.

https://en.wikipedia.org/wiki/Pigeonhole_principle

Also, even if your argument were valid, it only implies that metro area inequality matters. So why are you looking at national inequality?

Neither of those links/graphs show that inequality had decreased. His argument is based on the false assumption that one-hump is more equal than two.

Here are some graphs that argue the other way: http://www.newyorker.com/news/john-cassidy/pikettys-inequali.... Note that the decrease in inequality after WW2 is due to the New Deal and other massive social welfare programs that redistributed wealth. The moment we stopped doing that inequality started growing again.

The only way to counter the Matthew Effect is to change the feedback loop. The post-WW2 programs and post-civil rights movement Affirmative Action programs (coincidentally affirmed by SCOTUS today) are examples of doing just that. I'll not make an argument for the merits of desegregating society here, but the Parable of the Polygons (http://ncase.me/polygons) is another example of how deliberate counteractions are the only way to change the "natural" course of things. Human culture (including evolving social mores, laws and science) is a prime example of changing the natural course of things. Laissez-faire doesn't always work.

How do such decisions cause the Matthew Effect? In so many ways! But here's one to get you started: Black Americans are clearly less advantaged than white Americans. That translates to white Americans growing up in a safer environment, being better educated, having more connections to other advantaged people, not being constantly violated (I highly recommend you read Ta-Nehisi Coates's Between the World and Me) the way blacks are -- I could go on and on. So of course whites will more often be "better hires" from a pure business perspective. So more whites get hired for better jobs. So guess whose children get to live in better neighborhoods, get to go to better schools, and all the other advantages I listed earlier? That is the Matthew Effect.

Your polygon parable shows that phase segregation is a natural result of like-dissolves-like (something chemists have long known), and will likely occur naturally as a result of individual choices rather than malicious effects.

While this is interesting - it provides strong evidence that various racial disparities may NOT be the result of racism - it doesn't support the Matthew Effect at all.

Black Americans are clearly less advantaged than white Americans.

No one disputes this. The question is why this will result in differing exponential growth rates. You haven't provided a mechanism for this.

Indians and Chinese are also clearly more disadvantaged than black Americans - just think how much effort they need to go through simply to get the natural advantage of being legally permitted to work in the US. Yet their growth rate (in terms of life expectancy, income, etc) is higher.

This demonstrates that your "clearly less advantaged" theory does not imply differing exponential growth rates.

Also, your source about within-country inequality (i.e. Piketty) is silly. Even if we only assign 3/5 weight to Indians and Chinese people [1], inequality has still decreased. Any discussion of inequality which mentions Balzac more frequently than India/China/Africa (such as Piketty's book) is hopelessly myopic.

[1] Is there a reason that the Indian people around me (I'm in India) deserve less weight than Frenchies? I'd love an answer more detailed than "Trump 2016!!! WOO YEAH!"

The issue with this reasoning is that your "base rate" is sampled from the general population, while the population of Princeton graduates will look nothing like the general population. So the question is how informative is the base rate prior (from gen. pop.) when you have further information of Princeton graduate?

You may then suggest that since we do not have a meaningful prior based on Princeton grads, it makes sense to use the gen. pop. base rate. After all the Asian still has to pass a more rigorous process to attend Princeton. But this ignores the unique barriers that the black candidate also had to pass, for example economic hardships, lack of parental and peer examples of successful college grads, all sorts of cultural issues, etc.

Ultimately the question is how valuable is the gen. pop. prior given that we do know the candidates come from the population of Princeton graduates? At the very least its significantly reduced, while it may also be entirely worthless. Given these caveats and the dramatic social issues that result from using this prior to significantly bias your hiring process against black candidates, it seems like a moral imperative not to use it.

The base rate I'm using isn't the population base rate at all - it's the top 25 college students base rate, sliced by race.

Between 2/3 and 5/6 of black college students at the top 25 schools did not meet the usual intelligence/test score cutoffs to get in. Thus, the base rate for "exceeds normal cutoff" for a black student at Princeton is in the range of 1/6-1/3. (In contrast, the base rate for Asian students is likely higher. The base rate for athletes and the children of large donors is likely lower, for much the same reason.)

The various factors you discuss might be causes for why most blacks don't exceed the normal cutoff. But that doesn't mean that ignoring it will be an optimal decision for a person attempting to hire high intelligence/test score individuals.

If you're just going by entrance GPA/test scores, then I would still question the usefulness of that base rate after the person has graduated and also passed whatever screening process your company has. GPA/test scores surely correlate with intelligence, but they can be gamed to a large degree especially by students with the means and motivation (the typical Asian student). As far as I've seen GPA/test scores only correlate with success in the first year or two of college. So again we're left with the question of how meaningful is your base rate at all? I don't think there is any reason to think it provides any information that isn't completely overshadowed by the fact that the person graduated, their college GPA, and their success in your own screening. Using such dubious priors is biasing your decision making based on unjustifiable assumptions, and will likely end up with worse decision making.

See the math here:

https://news.ycombinator.com/item?id=11965961

You can play with numbers to see exactly how important the base rate is.

If you believe your screening process is 99.9999% perfect, then most likely base rates are nearly irrelevant. I don't think my screening process is that good.

The math isn't what I'm questioning. The math doesn't tell you about the quality of your prior, just how to factor it in. But if your prior is crud, then you end up making worse decisions as a result. By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening. This is not an "uninformed prior", you are explicitly modelling the belief of the relevance of entrance scores. Modelling no belief would be a uniform prior, that is no bias in the population.

By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening.

Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations, that's a perfectly valid assumption.

The argument that if the prior is wrong, it hurts you is of course true. Similarly a uniform prior in a non-uniform situation will hurt you - see this example for an illustration of how ignoring base rates can spread HIV. https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate...

The difference in the HIV case is that we have an actual population sample that justifies our chosen base rate. That is to say, this isn't a "prior belief", this is a measured statistic. It's applicability is precisely because the population of individuals getting an HIV test is random from the general population, from which we have a measured statistic. But the use of the base rate in this case does not justify the usage of informative priors in every case.

>Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations

But it might. The "population" of graduated + passed screening bears no a priori relationship to the population of entering freshmen precisely because this is a non-random sample, orders of magnitude smaller than the original population. Furthermore, there is further evidence that there is no relationship precisely because of the studies that show that entrance exams bare little correlation to college success past the first year.

Notice the framing. Much of this is framed well from the author's point of view- you agree, or you are wrong.

"Tost’s research shows, may help individuals recognize the privileges they enjoy as part of a dominant group, as long as they also believe they’ve experienced disadvantages as members of other, subordinate groups (and thus can identify with people who feel disadvantaged in comparison with them). White women overall, for instance, are more likely than white men to view themselves as beneficiaries of racial privilege; they get it because they, too, have had to deal with discrimination."

White men can't be part of a group that experiences discrimination? What if they're old or young? Have a disability? Are part of the certain ethnic or religious groups, e.g. Jews? Are gay, bisexual or have some other sexual orientation? Who's discriminated against isn't a competition. But all people are judged and get treated how they do in part because of superficial factors (and the prejudices associated with them). At least sometimes.

As referenced by another comment, this article was enlightening: research which suggests that many stereotypes e.g. around race or gender are actually useful priors and improve human decision making when you don't have other, better information:

http://www.rci.rutgers.edu/~jussim/unbearable%20accuracy%20o...

I noticed that right away. Things like "We believe we know good talent when we see it, yet we usually don’t—we’re terrible at evaluating people objectively"

Some are really good at evaluating talent/people. Author seems to want to lump everyone together in a mass of "we" to make some grand point. Missing the trees because of the forest.

Gotta say, I love how not once was age mentioned. If you want a different perspective, get it from someone who has been around for awhile. It would be interesting if someone tried to calculate how much suboptimal hiring costs the economy, I have this suspicion that failure to utilize people with real experience will top the list.

how much suboptimal hiring costs the economy

But it's not about the economy, it's to preserve the status quo for management, a small subset of the company.

While these cognitive roadblocks affect everyone, I wish the tech community and others who believe in "meritocracy" would take greater hede of this article and other criticisms of it.

Seems like a really small sample pool, 73 candidates.

I am bias on every decisions I make. I dont want to hire lazy people or people who stink or people who talk too much or busy bodies or on and on and on. Being a single entity among many makes you bias, just learn to be bias on the correct things.

If you read the article, it points out that people alter their bias threshold depending on who they are interviewing. Black guy is a little quiet? He's shy, a bad culture fit. White guy a little quiet? He's modest, a great culture fit.

Yes I read it, and your example I covered when I said be biased on the correct things. Not all people alter their bias but part of the problem with this article is its generalities that dont apply to everyone but are written about like they do.

That's like saying "If you're drowning, just start swimming".

Reply on news.ycombinator.com