UK passport photo checker shows bias against dark-skinned women
bbc.co.uk
bbc.co.uk
As a Swedish/Finnish man (and a somewhat stereotypical nerd at that), I'm paler than probably 95% of the world population.
Although facial recognition features (in the sense of trained AI models) often work fine on me given a good quality underlying photo, what often fails instead are preprocessing steps or color correction, especially in real-time systems. This can be the camera itself or software handling the image later.
For example if I'm in a Google Hangout in a normally lit room and then I move into slight sunlight or a lamp, my face will overexpose into a blob that looks like a low quality photo of the sun. It renders white, like #ffffff proper white. It's not actually overexposed at the hardware/camera level, but the software "corrects" it that way.
This typically never happens to other people.
At this points these complains are bullshit in my opinion. It shifts the debate about if we want to accept facial recognition to the side. It is like a free gift to those that sell these systems.
So if facial recognition is actually bad for minorities, people bringing up these topics should not be their friends. My opinion, I know it is a strong one, but it is also more correct.
Is a passport checker even needed for security would be a better discussion topic.
Your inconvenient experience differs at a material level, because as a white-skinned man, many other systems are optimised for you.
(1) https://www.theguardian.com/lifeandstyle/2015/apr/30/fda-cli...
It's a pity they didn't question the Home Office more on what they have done since 2019 to fix the bug since they knew existed from launch — e.g. how many times has the algorithm been retrained since then with good passport photos (since the issue still seems unresolved)
EDIT: Turns out from the New Scientist article linked that it actually launched in 2016, misread from the BBC one.
Also I'm not aware of any credible authority on the subject that would say "intent or a conscious prejudice" is any type of requirement for something to be considered racist.
From Collins[1]
>any program or practice of racial discrimination, segregation, etc., specif., such a program or practice that upholds the political or economic domination of one race over another or others
From Merriam-Webster[2]:
>the systemic oppression of a racial group to the social, economic, and political advantage of another
From Wikipedia[3]
>Institutional racism (also known as structural racism, state racism or systemic racism) is racial discrimination by governments, corporations, religions, or educational institutions or other large organizations with the power to influence the lives of many individuals. Stokely Carmichael is credited for coining the phrase institutional racism in the late 1960s. He defined the term as "the collective failure of an organization to provide an appropriate and professional service to people because of their colour, culture or ethnic origin".
Racism isn't purely about the belief that one race is superior to another. A system that oppresses one race in favor of another is racist. This example meets the definitions of all three sources.
[1] - https://www.collinsdictionary.com/us/dictionary/english/raci...
Your argument puts "poor training data", which I originally commented on, in the same bin as for example apartheid. Widen it to that use devalues the word.
I have made my point and have no interest to continue this discussion further.
Those first two quotes are literally dictionary definitions of racism. You are free to disagree, but you aren't just disagreeing with me. You are disagreeing with the dictionary.
Yes, the key word is "used". And the point why it isn't is because we pay now more attention to the target of racism than the perpetrator.
> You need to look at if there was an intent or a conscious prejudice.
Or alternatively, you need to look at what happens to the people being subjected to racism. Practically no one wants to be seen as a racist anymore. Intent and conscious prejudice is now hidden.
In my eyes, that is progress. We move forward from assigning guilt to trying to improve the situation for people subjected to racism.
I kind of doubt, that it will mean, more people would be happy or neutral being called a racists.
At what point can you say it's not racist, just not very good?
They offer no way to evaluate the statistical validity of the conclusions drawn from "BBC Research". They looked at 1000 politicians, but what was the color breakdown of those? What was the actual sample size of dark-skinned people? Who coded the photos' skin colors and how? This is just utter garbage without any rigor at all.
Their "BBC Research" indicates that male faces read much better than female, but they don't comment on this at all. Why did they choose the race angle and not the sex angle?
Moreover there are real reasons why dark-skinned features are harder to read than light-skinned. It's the same reason things are hard to make out in the dark, or you can't tell the features of an object covered in vantablack. It's physics. Less light means less information.
This whole article is the perfect distillation of low-effort motivated race baiting. To an astute reader it proves the opposite of its bias - if this is what is considered racial greivance worthy of breathless BBC articles, there must really be nothing significant going on to complain about.
There is zero chance that anything like this would ever have been published if the results went the other way, no matter to what degree.
"Kirstie Whitaker, programme lead for tools, practices and systems at the Alan Turing Institute, reviewed our code, independently reproduced our results, and provided input on the validity of the reported outcomes."
Would be interesting to see a proper write-up of the same study.
That said, let's say that it is taken as given that it's harder for a ML set-up to determine good photos for dark skin. Should it be used at all then? Is it acceptable that it's harder for the ~15% of the population to get a passport than the rest of the population because of an algorithm in the loop?
I don't think you can put it down to other factors like socio-economics as the Home Office reproduced this in their user testing before launch and acknowledged as a problem of the system (Vs the photos)
Hardly anyone takes their own passport photos. Only the very rich would get a professional photographer.
Rant start >
I think this just derails discussion about how system is really broken. My friend applied for UK residency around Christmas. She matches all criteria, her husband studied and now works in UK, she is EU resident. Border service holds her passport for 11 months now!!! Only info we can get is automated paid call, for 15 euro they tell us it is in progress...
Meanwhile she got pregnant and is about to deliver. All faster before border service can even answer single question... She is about to deliver, 2000 km from her husband, and can not even leave this country, because UK is holding her passport. And yes, she happens to be black as well...
...rant end
For any algo, I'm sure you'll find features where it underperforms, and since, well, photos are 90% race-determined, of course there will be racial differences.
Bureaucracy doing its best to not deliver what the law says it should is, meanwhile, blatant racism. Just sounds less exciting than a Skynet-style racist operation of white men forcing Asian women to join a different queue at the airport.
Not making your software function properly for all the legitimate input it has to deal with is a bias. Not working to eliminate that bias because it affects a group you don't happen to be part of could even be called racist.
You are arguing for the developer's convenience. Good software is about providing functionality to users, not developer's convenience.
Perfection is the enemy of good and many bugs get relabelled features (and no I am not doing that in this case, just pointing out that software development always involves trade offs).
And stop calling it racist. Racist used to be something pretty bad, now it's being applied to everything.
That doesn't surprise me even a tiny bit, given that the problem will already present itself in the camera - exposure settings are calibrated for white-ish people, and automated photo booths don't have a human to correct it or offer multiple different exposures to the user. Scanning the photo has the same problem.
> Photos of politicians were collected from parliaments around the world. The gender and skin tone were recorded for each photo using the Fitzpatrick skin-tone scale. Each image met the Home Office standards for a passport photo.
That definitely suggests the software used for automatic image quality check simply isn't up to par.
Her photo shows a clear-as-day mouth closed, which was repeatedly interpreted by the system as mouth open.
No exposure settings would change that outcome.
I always thought that it's just harder to extract facial features from an objectively darker image. Or is it real bias in the algo or the training set of photos?
Amateur photographer here. The key thing is to ask why the image of PoC is "objectively darker", and the answer to that is because exposure meters/auto programs in cameras are calibrated for correctly exposing white people.
joking aside - better testing might help
The same applies to computer algorithms. The real danger is when those algorithms are treated as without bias.