AI expert calls for end to UK use of ‘racially biased’ algorithms
theguardian.com
theguardian.com
If the software is accurate at picking choices that match the set goals, then it would seem that: the software is doing its job and is 'blameless', the racism is on the side of those that set the goals and/or the people claiming 'racism' because a person of a given skin colour was selected.
If it could be shown that there's low success in the software, yet it's relied on despite racial bias, then I would think few rational people could argue against its removal. (edit: s/for/against)
Edit: I suppose that I'm asking for better auditing of the results vs the desired outcomes. Race seems like a red-herring until it's shown that the metrics support poor performance of these systems.
The problem with most of these deep learning/AI systems is that they are more or less pattern detection systems. To an AI racial biases in its input data is just another pattern.
Surely we have a baseline that can be used, such as when these systems are not in play?
That's assuming that we can agree that the justice system works impartially to begin with. I don't know of many people (including myself) that could agree with that.
Well, no.
It's not enough to simply decline to input racial data into the machine-learning system. The system may discover a proxy for race, and make decisions based on that proxy. Name and address, for instance, may be enough to give a good idea as to someone's race.
We expect human judges and parole-boards, for example, to discount racial information, even if there are demonstrable correlations associated with race. If the machine-learning system is not built with that discounting in mind, it isn't going to do so of its own accord, as it was of course built to detect and exploit correlations.
> If it could be shown that there's low success in the software, yet it's relied on despite racial bias, then I would think few rational people could argue for its removal.
I don't follow. If the software has been unsuccessful, that seems like a good reason to remove it. If, additionally, it has been making decisions on racial grounds, that seems like another good reason to remove it.
> Race seems like a red-herring until it's shown that the metrics support poor performance of these systems.
As I hope I've made clear above, this isn't right. It's not just about performance, it's also about ensuring the system does not make decisions on the basis of an individual's race.
You've been denied parol because someone with a name similar to you skipped bail is unfair regardless of race.
You've been denied parol because you come from a rough neighborhood is also unfair, regardless of race.
In fact, even if 99.9% of all people named X skipped bail, it would be unjust to deny somone bail because his name is X.
Racially biased could mean two things: despite not taking into account race the algorithm groups results racially at a statistically significant rate, or the algorithm is outright using race as a factor.
I assumed the former because it was not made clear in the article what was being talked about. Half the article is about things that may/may not use racial information (it's not said), and half the article is about things that do.
Are we talking about the same thing here?
> I don't follow. If the software has been unsuccessful, that seems like a good reason to remove it. If, additionally, it has been making decisions on racial grounds, that seems like another good reason to remove it.
I mistyped there. I meant to say "few rational people could argue AGAINST its removal".
I had originally edited my sentence partially and mangled the meaning entirely.
Frequency of doctor visits seems a good indicator for determining how likely an insurance company will have to pay on a policy. If you want equal outcome, you end up setting a higher premium for everyone, and skip the underwriting altogether. The next part of the discussion would be subsidizing the higher premium for those that can't afford it. Just keep in mind that because there are fingers on the scale, fewer people can afford the "equalized" insurance than could afford the honestly evaluated coverage. (Prices are higher for everyone, so the pool of subscribers ends up being smaller if subscribers can opt out.)
It becomes a never-ending cycle of controls and impositions leading to collapse. Have the realistic underwriting, lower prices, and subsidize those (overall) lower premiums when needed.
Software is written by people. Racist software comes from racist beliefs. A belief that a neural network will learn objectivity is a belief in objectivity, and both are wrong.
Race is not a red herring; even though race may be only a social construction with no objective science behind it, the fact that people believe in and act as if race were real is a problem. In particular it often violates the Civil Rights Acts.
> auditing of the results vs the desired outcomes
The core problem is this assumption that the goal is to prioritize accuracy. When making "life-changing decisions ... in areas from job applications to immigration", many different goals need to be balanced. Who gets to define what "desired outcome" means?
We already have a hard time simply defining those goals. How would you even begin to define a cost function for e.g. "immigration" decisions when simply bringing up the topic is one of the easier ways to start a heated political argument?
> then it would seem that: the software is doing its job and is 'blameless'
The software is doing exactly what it was told to do: find a particular signal in some set of data (without explicit race. The problem is we spent several hundred years systemically embedding race into most aspects of society, which becomes a minefields of problematic local minima/maxima traps for your ML algorithm.
Let's say we have an good ML algorithm that provides some sort of evaluation about job applicants. Nothing related to race is included. The algorithm is used by a company in Chicago[1][3] or D.C.[2][3] hoping to hire the "best" people. Segregated populations means different schools/etc. Even if you didn't give the algorithm race data, most ML is practically guaranteed to find one of the many ways to infer race. Unintentionally, the ML algorithm ends up doing something similar to redlining[4], because that's what is in the data: the systemic effects of historical racism that still reverberate through society.
[1] https://www.washingtonpost.com/graphics/2018/national/segreg...
[2] https://www.washingtonpost.com/graphics/2018/national/segreg...
[3] red-dots="White", blue-dots="Black", yellow-dots="Hispanic", etc. See the full article for many more examples: https://www.washingtonpost.com/graphics/2018/national/segreg...
We should be upvoting more substantive writing, making serious accusations on racism should be backed up by real evidence, not quotes and opinions of someone parroting the author's narrative
Many of these pieces smell phony. I'm certainly not saying there isn't institutional racism at work here, but I think we need way more detail to evaluate these claims.