Hacker News Reader

Comment by cycrutchfield | Hacker News Reader

That’s gonna be a yikes from me, dog.

Edited this (since you did as well) to say that simply pointing to statistics and saying that “facts don’t care about your feelings” is sophomoric and probably intentionally disingenuous. For instance, statistically in the US black people are more likely to commit crimes than white people. So should we implement systems that target suspects based on the color of their skin? Obviously not. There is a very important confounding factor, socioeconomic status, which happens to be very correlated with skin color in the US due to a long history of racial discrimination. So saying that “these systems should not be racially biased” is an important criteria.

View on HN

> So should we implement systems that target suspects based on the color of their skin? Obviously not.

Why not? Why is that obvious?

Utility: if the system isn't effective, there will be additional crime and additional victims of crime ('''bias''' against those victims)

Group fairness: the system should target the population in a uniform manner across all skin colours (actual statistical bias to favour skin colours over-represented in crime)

Procedural fairness: the system should follow the same process for all skin colours ('''bias''' against skin colours over-represented in crime)

In a world where skin colour isn't uniformly distributed across crime rates, you can't have all of those. You must explicitly sacrifice one of those factors to have a chance of satisfying the other two. Ignoring this trilemma doesn't make it go away. No matter what you choose, including inaction, there will be a harmed party.

This was an actual scenario faced by the COMPAS parole sentencing system. There are no easy answers.

P.S.

You are surrounded by systems that target you based on the colour of your skin every day, and the designers of most of them aren't even aware of it. Scandals like 'Woman slams ‘racist’ Boots for putting security tags on hair products for black customers but not those aimed at white people'[1] are driven by very simple models and very limited data like [sku, shrinkage rate] that you would think can't possibly give away skin colour. And yet they do. And if you want to make those models 'not racist' then you do actually have to statistically bias them on skin colour. Again, no easy answers.

[1] https://www.thesun.co.uk/news/9941383/boots-racism-security-...

We already discriminate as of today based on gender statistics in areas such as car insurance, where men pay more on average, because they're statistically more likely to cause accidents and drive recklessly. Why can't statistics be extended to other areas?

In the EU exactly that kind of gender discrimination is illegal. Insurers are not allowed to charge men more than women, just for being men.

The question is, do you want to live in a society where prices are set based on one's genes. I sure don't.

So what the EU does is just charge more for everyone which means women pay higher rates for car insurance despite having lower accident rates. So in the goal of equality, women are being forced to pay more than they should based on their level of risk. That’s exactly the opposite of discrimination — women pay more despite lower risk, which makes that EU law, discriminatory.

In the age of ML this is almost impossible to police and very difficult to implement even if you want to.

You blind your model to gender but it learns to infer it from names. You blind it to names, it learns to infer it from other inputs. By the time you've blinded your model to anything that gives away gender, it no longer has utility. If you instead intentionally alter the model to still have utility but impact both genders equally, it's no longer procedurally fair and does in fact discriminate.

This is the trilemma mentioned above. You must choose which of those factors you care about and which one you'll sacrifice. If you're unable to make that choice and morally stand behind it, then probably you shouldn't be building said model/system in the first place.

This is one of the major topics of the ML ethics conversation.

>The question is, do you want to live in a society where prices are set based on one's genes. I sure don't.

Those genes do clearly make us behave in different ways, so why not?

I'm not a feminist, but it would be fairer for women to pay less for the same insurance since they are less likely to get in trouble with their cars. Why should they pay more?

I think the argument reduces to since we can get hyper accurate data on an individual level how can we treat people fairly? If an insurance algorithm charges one person who is higher risk 100x more than someone else is it really a fair form of insurance?

I think that's an interesting point, and I can't say I have a well thought out position on it, but I'm happy to have stumbled upon this conversation.

One one hand, I feel ok paying extra for my health care to cover others who might have been left out previously because of individual pre-existing conditions. Us pooling all together to take care of every citizen makes sense.

However in a purely hypothetical scenario where say men are 1000x worse at something than women, let's say driving, say car insurance for women is $100 and it's $100k for men, should we really charge women 50k just to make it fair based on the "we must ignore gender" principle? I realize in the real world there might be no such stark contrast between genders or ethnicities, in which case giving everybody the same higher rate would probably be more sensible.

Do you really think that the melanin content of your skin has any causal relation to how likely you are to commit a crime, or any other outcome?

Literally no one is treating race as if it was about "melanin content of [one's] skin", so that's a strawman if I ever saw one.

It's a very visible trait, strongly associated with ancestry, and thus associated with many other, more important, traits.

Well, police don’t perform actions based on the identity of your parents or grandparents. Skin color is evidently a pretty important feature in terms of determining the actions that police may take.

But, let’s follow your logic for a second. It seems you are suggesting that it is acceptable for insurance companies to ask you for the identity of your parents or grandparents to decide what rates to charge you. Does that sound like an outcome that you think would be acceptable?

Not necessarily (I haven't thought about the issue too much).

My point is more that, if insurance companies discriminate using any non-genetic, non-biological criterion, the outcome would still not be a distribution strictly representative of the population. Because people's choices are influenced by their background. For example, it's quite probable that the distribution of brand and type of car is not uniform across racial lines (or between men and women). Would you consider the outcome resulting from this to be biased?

In fact I'm pretty sure it's how European insurance companies still manage to charge more for men, despite being forbidden to explicitly ask that information. That information still affects the pricing model through things like occupation, etc.

Do you really think the melanin content of your skin has any causal relation to how well you can play basketball? Or maybe melanin content isn’t the only difference.

Well, not everywhere allows discrimination based on sex, some US states don't, for example (California just joined that club last year). Plus the EU doesn't either.

People generally find a discrimination "fair" if it's based on something that you can change whereas an "unfair" discrimination is typically something that you are.

Do they commit more crimes, or do they get caught committing more crimes? And is that because they are targeted more for searches?

The problem is that people still tend to complain about the outcome if it's not an egalitarian distribution, regardless of the accuracy of the inputs.

Let's say I want to identify violent criminals based on things like type of jewelry or clothes they wear, or tattoos (symbol, location). Reasonable model: it tries to identify gang membership by visible gang membership symbols (i.e. it exploits something gangsters themselves signal). It's not perfect (false positives: hip hop artists who are not necessarily actual gangsters) but it's probably a decent model. I would not be surprised if the outcome did not match the racial distribution of the US population, precisely for the reason you mentioned. Would that model be racially biased?

Reply on news.ycombinator.com