Hacker News Reader

Comment by visarga | Hacker News Reader

It was clearly a mistake. They don't act in a constructive way.

Let's take Timnit's paper for example. She found out that Google had been using a biased language model in search. The bias was like - 'male' association with 'doctor' as opposed to 'woman' association to 'nurse'. But she didn't show how this is harming anyone in a concrete way. Just theoretical. And then she offered no solution, just blaming the work of others, using her paper as a soap box to raise scandal and make herself holier than thou.

View on HN

While I am not a fan of Gebru's handling of her own firing, this is a mischaracterization of her work.

She routinely offered suggested methods, experiments, and even new datasets [1] that fixed what she saw as wrong. She does have a fair bit of practical ML experience as shown in the computer vision papers in her publication list.

1. Buolamwini, Joy, and Timnit Gebru. "Gender shades: Intersectional accuracy disparities in commercial gender classification." Conference on fairness, accountability and transparency. 2018.

Of course I was aware of her Gender Shades paper. It's a small bias evaluation benchmark dataset.

> We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments.

A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve.

A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?

With the hard realism approach, a corporate entity would fund an independent body of research, and then re-use publications that match their agenda.

Hiring someone (perhaps with aspirations to buy their loyalty?) entails risks. Instead of pulling funding from the "independent think-tank", google now finds itself in the midst of a potential discrimination/political scandal involving an employee.

> But she didn’t show how this is harming anyone in a concrete way.

To roll with the example “man” ~ “doctor”, “woman” ~ “nurse”, the harm is having a giant and widely used search engine reinforce baseless gender biases, ie that there is no underlying reason why women should be nurses and men doctors. What is the harm you may ask? The harm may be subtle, eg being surprised when you find out your next doctor is a woman or your next is a man. It could suppress career choices and aspirations, and it could even be financial, eg reinforcing systemic pay gaps.

This kind of hypothetical effects should be documented in concrete cases by a good ethical scientist instead of just described from imagination. For example, when someone searches for a doctor, let's say a male comes up first - who stops at the first Google result? They would probably need to go deep and read about the doctor's experience and find patient reviews.

Part of the issue is that the harms of gender bias (and other types of bias) should not need to be made explicit, but part of the research canon. Should a security researcher outline the harms of an attacker obtaining user credentials, or is our imagination sufficient because the harms are well known to us? And if you were looking for more in depth studies, then there is a ton of published research, maybe not all of it on arxiv or in machine learning journals.

At some point imagination has to make touch with reality otherwise it can become unhinged. Yes, security researchers can enumerate concrete cases where "the harms of an attacker obtaining user credentials" caused damage.

That particular "bias" isn't actually a bias, it's an accurate learning about the distribution of genders between jobs in the real world. Very few professions have an exactly 50:50 balance of men and women. Most are tilted towards one gender or the other. The purpose of a correct search engine is not to reduce my "surprise" at arbitrary events but rather to give me the information I'm looking for, which will more often than not be questions about the real world - not the fever dream of some hard-left activist.

Indeed. To intentionally skew the data such that, for example, men are over represented as nurses, is in fact introducing bias to the data based on prejudice.

You’ve essentially created a fictional data set because it’s biased due to the underlying prejudice (preconceived opinion that is not based on reason or actual experience) that men ought to be nursing more, despite that not being reality.

We’re in a strange situation where we have large concerted efforts by activists to inject fiction in to our facts (whatever the medium) with the aim of distorting perceptions in such a way as to some how correct what they perceive to be injustice in the real world.

Reply on news.ycombinator.com