Hacker News Reader

Comment by headalgorithm | Hacker News Reader

From the article:

"Those disparities can sometimes be a matter of life or death: One recent study of the computer vision systems that enable self-driving cars to “see” the road shows they have a harder time detecting pedestrians with darker skin tones."

Just think about the consequences of deploying such systems.

View on HN

I wonder if that's also true for human drivers. I certainly have a harder time detecting pedestrians wearing darker clothing, at least in some circumstances.

That's why road workers, cyclists, and joggers wear brightly colored reflective vests.

There is also an important ethical difference between dark skin tones and dark clothing.

I see the point you are making, but to the extent that an AI can do better than a human due to the physical aspect, for example, of dark skin reflecting less light at night, I think we should at least try.

So maybe we should not be using opaque ML algorithms for life-and-death situations to begin with? Instead of, you know, hoping that whomever is screwed by them has an active advocacy group in places like MIT.

Using NL for such things hasn’t been working too great, so it seems worthwhile to try for something better.

such as?

Are you asking for examples where NL hasn’t done well or for potential ways to do better?

the latter. It’s relatively easy to say “stop doing X”; it’s harder to say “...and have you considered Y instead?”

I’d hope it was obvious from context that ML was the potential way to do better here.

I read you as saying that it was the thing doing poorly and that some superior object detection method was required. What that approach could be is totally unobvious to me.

Maybe you missed my too-subtle use of “NL” to describe natural learning i.e. squishy meat computers.

The story doesn’t cite the specific paper. But I think I submitted the paper last month:

“Predictive Inequality in Object Detection”

https://news.ycombinator.com/item?id=19379827

https://arxiv.org/abs/1902.11097

I’m not denying the conclusion of the paper, but it does have a lot of limitations. Also, Tesla notwithstanding, isn’t object detection primarily done via lidar?

The computer vision systems could still beat real human vision systems. This could be by absolute numbers crashed into, or by having less bias than humans, or by both.

It's not even a bias problem. It's a lighting, contrast, and camo problem. You could as well claim a bias between asphalt and concrete pavement, based on what skin tones are more visible with the pavement as a background.

Suppose the system does beat real human vision by absolute numbers , but it turns out that it is objectively worse for someone who is not white, and the higher average is because the majority of people in that society are white. Is that acceptable?

> The computer vision systems could still beat real human vision systems.

For pattern recognition, No, the computer vision system can't best a human.

But the entire self-driving system can. It is never tired, or distracted, has far fewer blind spots, and possibly LIDAR and IR to help it.

It can’t today, and until it proves to do so making claims about future tech in relation to present policy is a bad idea.

I'm not saying it does today.

I'm saying it could do so, while still having bias. The existence of bias doesn't make the computer worse than humans.

I saw a talk by one of the authors of the pedestrian detection study that I think they are talking about, and their results did not seem support the scary racist self-driving car hypothesis that the study clearly implies (only marginal/statistically insignificant effects, with very little controls for confounding factors (I believe they controlled for bounding box size or something, but very little else)).

With that said, fairness in ML/AI is a real problem, and some people are doing some really good/important work in this area. I am not familiar with Buolamwini's work, but I'm much more inclined to believe disparities in facial recognition than pedestrian detection where very little skin is visible and almost everything you're seeing is clothing.

"Just think about the consequences of deploying such systems."

Those systems are already widely deployed - they're called 'humans'. All vision systems are going to be more likely to confuse like colours.

This problem is quite a bit different from the 'mugshot' problem, or a problem wherein there was a bias or lack of training data for certain samples.

The author I think conflated a lot of the issues and just boiled it down to 'evil AI' which I don't think is the right thing to do.

Issues of ethnic orientation are quite a bit different from systems that have trouble literally due to the colour of something.

Reply on news.ycombinator.com