New York Regulator Probes UnitedHealth Algorithm for Racial Bias | Hacker News Reader

New York Regulator Probes UnitedHealth Algorithm for Racial Bias

wsj.com

This 'algorithm has bias' kind of headline needs to die.

Every system has '''bias''' in the colloquial sense so long as the inputs to that system are not uniformly distributed. The only thing the designer of the system can do is make a conscious choice about which way to '''bias''' the system and be able to justify that choice.

In practical terms it means that systems that deal with diverse populations are always going to be 'unfair' towards some group of people. The only control we have is who that group of people will be. This is inherent and unavoidable. We need to accept this fact and keep it in consideration when designing such systems (or choosing not to design them in the first place).

{procedural fairness, group fairness, utility}. Pick two.

https://www.youtube.com/watch?v=Zn7oWIhFffs

It comes down to the world frame some folks have. To them the world should have total fairness between all different people irregardless of predisposition, skill, situational luck, or effort put in. If it is not fair by their system they can fix it by pressing down on the scale. If any system or person has a fact that disrupts their worldview they are labeled things like "biased/racist/patriarchal/etc". Their desire for absolute equity requires rejecting facts that show people are different.

To people with this worldview an algorithm will always be "biased" because it simply reflects statistics instead of their view of they see as "right" which is with their benevolent finger on the scale.

I edited this slightly to try and remove any us vs them rhetoric. I am not passing any judgement just stating what I have observed.

That’s gonna be a yikes from me, dog.

Edited this (since you did as well) to say that simply pointing to statistics and saying that “facts don’t care about your feelings” is sophomoric and probably intentionally disingenuous. For instance, statistically in the US black people are more likely to commit crimes than white people. So should we implement systems that target suspects based on the color of their skin? Obviously not. There is a very important confounding factor, socioeconomic status, which happens to be very correlated with skin color in the US due to a long history of racial discrimination. So saying that “these systems should not be racially biased” is an important criteria.

We already discriminate as of today based on gender statistics in areas such as car insurance, where men pay more on average, because they're statistically more likely to cause accidents and drive recklessly. Why can't statistics be extended to other areas?

In the EU exactly that kind of gender discrimination is illegal. Insurers are not allowed to charge men more than women, just for being men.

The question is, do you want to live in a society where prices are set based on one's genes. I sure don't.

So what the EU does is just charge more for everyone which means women pay higher rates for car insurance despite having lower accident rates. So in the goal of equality, women are being forced to pay more than they should based on their level of risk. That’s exactly the opposite of discrimination — women pay more despite lower risk, which makes that EU law, discriminatory.

In the age of ML this is almost impossible to police and very difficult to implement even if you want to.

You blind your model to gender but it learns to infer it from names. You blind it to names, it learns to infer it from other inputs. By the time you've blinded your model to anything that gives away gender, it no longer has utility. If you instead intentionally alter the model to still have utility but impact both genders equally, it's no longer procedurally fair and does in fact discriminate.

This is the trilemma mentioned above. You must choose which of those factors you care about and which one you'll sacrifice. If you're unable to make that choice and morally stand behind it, then probably you shouldn't be building said model/system in the first place.

This is one of the major topics of the ML ethics conversation.

>The question is, do you want to live in a society where prices are set based on one's genes. I sure don't.

Those genes do clearly make us behave in different ways, so why not?

I'm not a feminist, but it would be fairer for women to pay less for the same insurance since they are less likely to get in trouble with their cars. Why should they pay more?

I think the argument reduces to since we can get hyper accurate data on an individual level how can we treat people fairly? If an insurance algorithm charges one person who is higher risk 100x more than someone else is it really a fair form of insurance?

I think that's an interesting point, and I can't say I have a well thought out position on it, but I'm happy to have stumbled upon this conversation.

One one hand, I feel ok paying extra for my health care to cover others who might have been left out previously because of individual pre-existing conditions. Us pooling all together to take care of every citizen makes sense.

However in a purely hypothetical scenario where say men are 1000x worse at something than women, let's say driving, say car insurance for women is $100 and it's $100k for men, should we really charge women 50k just to make it fair based on the "we must ignore gender" principle? I realize in the real world there might be no such stark contrast between genders or ethnicities, in which case giving everybody the same higher rate would probably be more sensible.

Do you really think that the melanin content of your skin has any causal relation to how likely you are to commit a crime, or any other outcome?

Literally no one is treating race as if it was about "melanin content of [one's] skin", so that's a strawman if I ever saw one.

It's a very visible trait, strongly associated with ancestry, and thus associated with many other, more important, traits.

Well, police don’t perform actions based on the identity of your parents or grandparents. Skin color is evidently a pretty important feature in terms of determining the actions that police may take.

But, let’s follow your logic for a second. It seems you are suggesting that it is acceptable for insurance companies to ask you for the identity of your parents or grandparents to decide what rates to charge you. Does that sound like an outcome that you think would be acceptable?

Not necessarily (I haven't thought about the issue too much).

My point is more that, if insurance companies discriminate using any non-genetic, non-biological criterion, the outcome would still not be a distribution strictly representative of the population. Because people's choices are influenced by their background. For example, it's quite probable that the distribution of brand and type of car is not uniform across racial lines (or between men and women). Would you consider the outcome resulting from this to be biased?

In fact I'm pretty sure it's how European insurance companies still manage to charge more for men, despite being forbidden to explicitly ask that information. That information still affects the pricing model through things like occupation, etc.

Do you really think the melanin content of your skin has any causal relation to how well you can play basketball? Or maybe melanin content isn’t the only difference.

Well, not everywhere allows discrimination based on sex, some US states don't, for example (California just joined that club last year). Plus the EU doesn't either.

People generally find a discrimination "fair" if it's based on something that you can change whereas an "unfair" discrimination is typically something that you are.

The problem is that people still tend to complain about the outcome if it's not an egalitarian distribution, regardless of the accuracy of the inputs.

Let's say I want to identify violent criminals based on things like type of jewelry or clothes they wear, or tattoos (symbol, location). Reasonable model: it tries to identify gang membership by visible gang membership symbols (i.e. it exploits something gangsters themselves signal). It's not perfect (false positives: hip hop artists who are not necessarily actual gangsters) but it's probably a decent model. I would not be surprised if the outcome did not match the racial distribution of the US population, precisely for the reason you mentioned. Would that model be racially biased?

> So should we implement systems that target suspects based on the color of their skin? Obviously not.

Why not? Why is that obvious?

Utility: if the system isn't effective, there will be additional crime and additional victims of crime ('''bias''' against those victims)

Group fairness: the system should target the population in a uniform manner across all skin colours (actual statistical bias to favour skin colours over-represented in crime)

Procedural fairness: the system should follow the same process for all skin colours ('''bias''' against skin colours over-represented in crime)

In a world where skin colour isn't uniformly distributed across crime rates, you can't have all of those. You must explicitly sacrifice one of those factors to have a chance of satisfying the other two. Ignoring this trilemma doesn't make it go away. No matter what you choose, including inaction, there will be a harmed party.

This was an actual scenario faced by the COMPAS parole sentencing system. There are no easy answers.

P.S.

You are surrounded by systems that target you based on the colour of your skin every day, and the designers of most of them aren't even aware of it. Scandals like 'Woman slams ‘racist’ Boots for putting security tags on hair products for black customers but not those aimed at white people'[1] are driven by very simple models and very limited data like [sku, shrinkage rate] that you would think can't possibly give away skin colour. And yet they do. And if you want to make those models 'not racist' then you do actually have to statistically bias them on skin colour. Again, no easy answers.

[1] https://www.thesun.co.uk/news/9941383/boots-racism-security-...

Do they commit more crimes, or do they get caught committing more crimes? And is that because they are targeted more for searches?

You're right, in that there is a tension between systems behaving equally and equitably, and that either one can be seen as "bias". (Although there can of course be systems that by design behave neither equally nor equitably, whether intentional or not.)

I think the "algorithm has bias" type of headlines proliferate because there's a common misconception that algorithms are somehow "objective". See for example Ryan Saavedra implying algorithms can't be racist because they're "driven by math": https://twitter.com/realsaavedra/status/1087627739861897216?...

It is not possible to avoid racial bias. If you make a model predict equally for two classes, you’ve likely only made your predictions worse for both.

Prediction outputs come with a degree of confidence. It is much better to change how you evaluate your predicted probabilities for each protected class than to try and force the probabilities themselves to come out even.

The algorithm is rarely going to be the problem. It is the input data that is allowed into the model. If you have a problem with how healthy whites are getting recommended for treatment more than sick blacks, you need to remove the inputs that otherwise differentiate them.

Outreach to healthy people is a different care than for someone who is sick. The comparison should be between two groups with the same need/types of care.

That’s backwards. If the problem is that the algorithm says healthy white gets treatment more than sick black; that means the algorithm has something else that it’s considering besides healthy and sick. If that’s the problem you’re solving for, you need to remove the input that creates the difference. Once it’s gone, the algorithm will be unable to produce a different result for two people with the same sickness level.

http://archive.is/AOYcV

Garbage in, garbage out. An algorithm trained on data from a bigoted society like ours will return bigoted results.

You can try to fix the algorithm but the only thing that can really work is to fix society...good luck with that.

>The probe follows a study, published in the journal Science on Thursday, that found that an algorithm sold by UnitedHealth’s Optum unit ranked white patients with fewer chronic diseases and healthier vital signs the same as sicker black patients.

>[...]

>“This compounds the already unacceptable racial biases that black patients experience

The article makes it sound like blacks are being discriminated against, but it looks like they're given preferential treatment? Healthy whites are ranked the same as sick blacks. That seems like a good thing, because less sick = lower premiums. Am I reading this correctly?

The very first sentence:

> New York’s insurance regulator said it is launching an investigation into a UnitedHealth Group Inc. algorithm that a study found prioritized care for healthier white patients over sicker black patients.

prioritized care for healthier white patients

How does an insurer prioritize care for members relative to each other?

From the article, “Dozens of hospitals and insurers in the U.S. use the Optum tool to identify diabetes and other chronic-disease patients who should receive extra assistance such as home-care visits, help managing medicines and support coordinating doctor appointments.”

My insurer sent me letters about something like that. I assumed it was of no value, just something they do to look like they care. Kind of like the card they sent for discounts on non-prescription "health related" pharmacy products. Like the homepathic cold remedies.

I also assumed that they weren't deciding who could receive the services, just who would get the letters.

...and I also assumed they didn't have my race recorded anywhere now that I think of it.

Perhaps by deciding whose claims are going to get paid, versus who gets shafted.

> The software used to predict patients’ need for more intensive medical support was an outgrowth of the Affordable Care Act, which created financial incentives for health systems to keep people well instead of waiting to treat them when they got sick. The idea was that it would be possible to simultaneously contain costs and keep people healthier by identifying those patients at greatest risk for becoming very sick and providing more resources to them. But because wealthy, white people tend to utilize more health care, such tools could also lead health systems to focus on them, missing an opportunity to help some of the sickest people.

https://www.washingtonpost.com/health/2019/10/24/racial-bias...

Reply on news.ycombinator.com