Comment by Eyas | Hacker News Reader

Comment by Eyas | Hacker News Reader

> Database contains photos of half of US adults without consent, and algorithm is wrong nearly 15% of time and is more likely to misidentify black people

I think technology is often a very effective way to describe what "systemic racism" is.

Racism propagates easily, and race-blind / race-agnostic / race-indifferent behavior does not combat it or counteract it.

Imagine a majority-white tech team, largely color-blind, yet still (by virtue of economic and geographic segregation) in a mostly white-centric bubble. The training set might be largely white. If some preprocessing or feature-detection is taking place on the raw images, it might be tracking features that could have higher variability among white people while overlooking other features.

If you don't know to go out of your way to check for, include tests, and give thought to mitigating imbalances caused by race (because you "see no race") then you allow this problem to propagate. In the case of computer vision, you will inherit the "racism" of every library you import from.

This reminds me of gender biased found in Word2vec[1]. It is hard to accuse the dataset of being sexist, especially when viewing it as a reflection of language in news article. Yet, people using Word2vec use it as a representative of language, not a representative of language in news, with all the baggage in today's (and past) culture. As a result, sexism can propagate, autocomplete and query suggestions might be biased. Then spending habits might be biased, etc.

  [1]: https://www.technologyreview.com/s/602025/how-vector-space-mathematics-reveals-the-hidden-sexism-in-language/

In my naive mind, I can't help but think it might simply be a case of the contrast ratio being greater in lighter skinned people - therefore making the hit ratios higher. Greater contrast ratios will make facial features stand out more.

Exactly my thinking. I worked with a company that developed on a medical device that took subcutaneous readings extracutaneously, and despite being developed almost entirely by Asian engineers, it got the best data quality from fair-skinned Caucasians, and the worst from dark-skinned Africans, as it totally relied on the transmission of light through skin tissue.

Claiming racism without understanding the problem is both unfair to the people involved, and an unwise undermining of one's own credibility for the times when people really are acting in bad faith. That may be in this case, but that may not be, and I don't like guilty-until-providen-innocent no matter whom it's leveled against (even a Federal agency of which I'm not the world's biggest fan, and a database the likes of which make me think we need a constitutional amendment to protect the privacy of the citizens).

Even if it was no-one's intent that the system is racially biased, allowing such a system to continue operate after learning of such a bias is a racist act.

I'm inclined to agree with you, but what would you say if it turned out that after much effort the engineering team was unable to make performance equal across all skin tones?

You mean that it would work better with darker skin?

What if the technical challenges of identifying a person's face mean it will always be challenging to identify someone when the light and/or contrast levels are lower? What then? Manually cripple the system and misidentify white suspects, too?

This post's comments are a great example of why technologists are fairly terrible at engineering social solutions, much the same way sociologists tend to be terrible at engineering technical solutions.

Ignoring known technical deficiencies in the system and assigning the same confidence levels in the output regardless of race could work out to be pretty racist, but what are they supposed to do, manually cripple the accuracy of the system down to the lowest common denominator?

In the medical device I mentioned above, it was medically useful for caucasians and lighter-skinned Hispanics, so-so with Asians, and almost useless for Africans.

So, what–don't go to market until the technology has parity across all skin tones?

Not everything is about intent.

I hate to be a pedant here...

but I actually wrote a pretty successful 3d radiology/cardiology package. So I understand medical devices.

I just wanted to hop in here and say that there is NO WAY the FDA would give you clearance to use the device you describe on African Americans under the circumstances that your posts have postulated.

It seems a no brainer that the facial recognition software should not be used on African Americans either. At least, if it was MEDICAL software... it would DEFINITELY NOT be cleared with that kind of a failure rate. And any provider or delivery network using the facial recognition software on African Americans would be on his or her way to prison.

But the regulations on legal software may be more lackadaisical than the regulations on medical software. Getting things exactly right may not be as important. And the penalties for misuse of medical devices are, without question, far more draconian.

> I just wanted to hop in here and say that there is NO WAY the FDA would give you clearance to use the device you describe on African Americans under the circumstances that your posts have postulated.

The FDA has approved drugs that are specifically intended to target medical issues in specific ethnicities, so I'm not sure about your assessment. That said, this product's market was outside the scope of the FDA, so it's irrelevant.

> It seems a no brainer that the facial recognition software should not be used on African Americans either.

Any data source which produces data that's better than a random guess should be considered when trying to solve a problem. That it offends some people's social sensibilities is a separate issue.

Look, I don't like the mere existence of the FBI's system, and I don't think the public's continued lack of concern for placing constitutional restraints on systems like it is a wise course forward, but to say it's a "no-brainer" when it might produce valuable leads is to focus on one aspect of the problem without considering the rest of it.

Just as it would also be a mistake to consider it more accurate than it is when trying to identify dark-skinned suspects. Witness accuracy is not a new problem.

It's also worth noting that the human eye is just as bad at facial recognition when the light and/or contrast is lower.

Not a matter of social sensibilities man...

the system doesn't work. At least not for african americans. You can't "socialize" away the fact that it doesn't work.

Use it for the races that it works for. DON'T use it for the races it does NOT work for. That is common sense... also known as a "No Brainer".

If it's better than a random guess–which it is, majorly–then it could constitute a valid data point in narrowing a search. There's nothing racist about using it to the level of accuracy it has demonstrated. If the system worked to the same degree of accuracy on Caucasians as it does on Africans nobody would be calling it racist, they'd be calling it a tool that can help build a list of potential suspects to further investigate.

To argue anything else is making it a matter of social sensibilities.

Only it's not better than a random guess.

Because in Eyewitness identification, you're never truly dealing with "random" guesses. Again, my experience is on the medical side of this whole thing. I know, and understand, the rules as my company is obliged to play by them. And there is just no question that we would not be allowed to use that system on african americans. Review boards would look on it the same way you might look on someone using leeches in the middle ages.

Only worse... because we KNOW it doesn't work.

Depends on the camera settings too. Older movies with mostly white casts messed this up a lot, by not properly calibrating for the darker skinned characters. Conversely, movies with all black casts get this right: they tune their camera/color settings to make the faces more distinguishable.

I ran into this a few weeks ago where I took a picture of a couple friends. The camera decided to pick the friend with dark skin, resulting in him looking great, and my white friend (who isn't even super pale or anything) getting washed out.

Yeah, I shoot semi-manual on my old D700 SLR. If you want high contrast ratios in dark areas of an image, everything else will be blown out - and you'll have to use a longer shutter speed - or suffer iso degradation or blur.

> If you want high contrast ratios in dark areas of an image, everything else will be blown out

Conversely could you say that in order to capture subjects with lighter skin, you need to increase lighting in the background or else the background will clip?

Here's an article about the cinematography and technology choices: https://www.washingtonpost.com/entertainment/movies/12-years...

> he was able to photograph black actors of dramatically different skin tones in a nighttime interior scene using just everyday house lamps, thanks to a sophisticated digital camera. “I just changed the wattage of the bulb, used a dimmer, and I didn’t have to use any film lights. That kind of blew me away,” Patterson says. “The camera was able to hold both of them during the scene without any issues.”

It seems like there might have been tech limitations initially, but with today's technology you can light your backgrounds and expose for either dark or light subjects well. The big challenge is having a photo with 2 subjects in the same exposure who you want to both look good.

Edit: by the way I too have an old Nikon camera. I usually shoot in Aperture priority with some intentional underexposure, and then edit it up in post :)

If it's a machine learning system (which virtually all face recognition systems are as far as I know) it depends more than anything on the training data.

It's just the same as with people: https://en.wikipedia.org/wiki/Cross-race_effect

Reminds me of the soap dispenser that doesn't detect black skin. [1]

1: https://mic.com/articles/124899/the-reason-this-racist-soap-...

Or cars and airbags being tested on male dummies for the longest time, resulting in women (who are shorter and weigh less) getting seriously injured/killed at higher rates in accidents: https://www.bloomberg.com/view/articles/2012-08-22/why-carma...

> In some cases, according to tests with female mannequins, small women were almost three times as likely as their average male counterparts to be seriously injured or killed. A study of actual crashes by the University of Virginia’s Center for Applied Biomechanics found that women wearing seatbelts were 47 percent more likely to be seriously injured than males in similar accidents.

There was a computer system featured once on Beyond 2000 that could recognize and mimic your facial expression.

But it was developed by Japanese and tested only on Japanese people. The host had to wear a black wig to make it work.

The Xbox camera also had trouble tracking African Americans.

That is a very interesting point regarding Word2vec, but it's inappropriate to call this example racism based on the evidence provided.

Only 16% of the US population is Black, while 70% is white. If the error rates between Caucasians and Blacks were the same that would, in fact, be racist - or at the very least undemocratic.

It's like asking that everything in the world be symmetric when only 10% of the world is left handed.

I don't see why being in a minority means accepting a higher chance of being confused by a criminal. This is how cycles of oppression propagate.

Those aren't equivalent unless humans are out of the loop entirely, which they shouldn't be in any area of law.

But then the humans can't be "color blind". They must explicitly go out of their way to undo the effects of the rest of the system by, E.g. Displaying extra care when dealing with a black suspect.

So, the usual, then?

Your chance of getting confused BY a criminal is probably more connected to your ability to comprehend and communicate.

Perhaps you not understand the metaphor. Developing takes time and money to produce accuracy. You're advocating greater per-capita investment for an ethnic group. That's racial discrimination. You are advocating racism.

It should come as no surprise that there's a difference. It is therefore suspect that they avoid mentioning the magnitude of the difference; it must not support their case otherwise they would highlight it.

Critical reading, logic, and reason will help protect you from this type of manipulation and inadvertent racism.

Your conclusion doesn't follow from your premise, which leads me to wonder whether you understand your own argument.

My camera's eye and face detection are also pretty racist. Works well with white and asian people, not so much with black people. It must have been written by women though because it recognizes women much more than men.

Trouble dealing with beards maybe?

Thank you for sharing that article. However, I don't see how the algorithm is itself racist.

That is like, if I give you glasses to put on, and through those glasses, you observe racism. That doesn't mean that the glasses made racism appear. And I don't think attaching blinders so that you can't see the racism would be helpful, either.

Word2vec is a lens that observes human output. And people saw things that they didn't want to see.

This is the thing about "systemic" racism. No one thing is racist. No need for racist intent. Just a combination of systems producing an outcome unfair to a particular group.

Could a disease be racist if it only targeted one ethnic group?

If it was created by humans then that would constitute a racist act. This system was created by humans.

No disease is. No algorithm is. No specific policy is. Systems can exhibit systemic racism. Other than hateful extremism coming from an individual, most forms of injustices are emergent properties.

In a hypothetical world where evolution doesn't care about morals and therefore doesn't make sure to only create differences in morally insignificant features, how would you be able to tell what is "systemic discrimination" and what is the state of genetic differences across groups?

Is the second also systemic discrimination? Can Joe the Down syndromer talk about systemic discrimination in having no chance at being the POTUS?

Facial unlocking programs have this problem too. Just search for "facial unlock racism" and there's pages of results from various sources and years.

Software is subject to the same effects institutions of racism, sexism, etc. and is a good example of just how pervasive these institutions are.

I call bullshit.

It's physics, not racism. Darker colors absorb more light and reflect less. In a photo the contrast is difficult to bring out with darker subjects.

I know this because I had a black dog and all her pictures came out like crap until she got old and turned salt and pepper.

Some people just have to see racism everywhere. That's too bad because it takes meaning away from real racism.

I'm pointing out a statistical fact from the article that black people are more hurt by this technology. I'm not arguing for racist intent, but against accepting unfairness towards one group as acceptable.

Dynamic range is of a photo is something controllable by humans building this technology. Contrast is a technical limitation and not a physical one. Lookup the articles on how early stage photography couldn't capture darker skin colors. Again, not racism, but related to incomes and advertising, but still had effects that disproportionally helped certain groups and harmed others.

Reply on news.ycombinator.com