Fearful of bias, Google blocks gender-based pronouns from new AI tool
reuters.com
reuters.com
From the article:
> Gmail product manager Paul Lambert said a company research scientist discovered the problem in January when he typed “I am meeting an investor next week,” and Smart Compose suggested a possible follow-up question: “Do you want to meet him?” instead of “her.”
So really, this is a fairly reasonable decision whereby it doesn't infer gender from signals such as profession.
The article doesn't address this, but I wonder what it would do given an explicit gender, e.g. "She's the investor I am meeting next week."
Is the idea just to eliminate all useful suggestions that contain genders?
Say do you want to meet them? would work fine in this case.
They is not singular. Replacing gendered pronouns with 'they' shits the flow of the sentence and creates ambiguity. There's a whole class of expressions that just stop working without a singular pronoun, hence the proposed 'ze' or whatever it is these days.
And what do you propose for all the languages where nouns and verbs and adjectives are gendered? In Russian, using gender-neutral phrasing when speaking about someone is an insult that implies that they're a thing rather than a person.
> Anecdote time: a friend from one of the nation's major universities was involved in a road authority funded study to evaluate the safety impact of newly installed speed cameras at several intersections. [...] They found there was none.
[1] https://news.ycombinator.com/item?id=18532386
Edit: after I replied, you edited your comment to add the caveat "except sometimes in 3rd person". "They" is always a third-person pronoun, and there are no criteria that would make it acceptable only in a subset of situations.
Edit 2: now the caveat is gone. Can you just reply to me rather than changing what I'm replying to retroactively?
My them example was in reference to the parent's phrase, not a general prescriptive. And in your example, while them would be a bit odd, from context you can gather them refers to Vincent alone. If there were further people one would likely be saying Vincent and his children are in the lobby, do you want to meet them now? In either case, I was primarily referring to they.
> And what do you propose for all the languages where nouns and verbs and adjectives are gendered?
I don't. I was explaining they usage that a lot of english speakers seem to not be aware of. I agree with this linguist: https://slate.com/human-interest/2018/08/singular-they-prono...
"Do you want to meet the <noun>?"
"Do you want to meet <name>?"
"Do you want to meet ___" <prompt for autocomplete from set of pronouns>
etc.
I'm not going to repeat myself - my opinions on that topic are recorded in the comments of the article about Richard Stallman's opinion of "singular they".
People use English this way constantly and it's well understood.
But that seems unlikely - we're probably talking about an engine that just recommends whatever occurs frequently in its corpus (hence the article talking about the team black-listing swear words and so forth). In which case adding gendered pronouns to the list seems reasonable enough, and non-gendered forms will presumably show up regardless (to whatever extent they appear in the corpus).
I'm struggling to tell from the article if Smart Compose ever actually did that. It's certainly implied, but the explanation given about gender balance in different fields is offered by the reporter. The parts actually attributed to Googlers only say they're being cautious about gendering in their outputs and couldn't find a fully-acceptable solution. Which leaves me curious they found specific semantic biases or instead noticed general issues like "our corpus taught the NLG to use a masculine default and simple fixes like randomizing pronouns created new issues".
(Either way, it seems like a reasonable decision. I'm just wishing for a more technical breakdown out of engineering curiosity.)
The other interesting point I saw is that Smart Reply apparently runs on the same system as Smart Complete. Since that offers fully-formed ideas, I imagine gender issues there could be vastly more embarrassing than single-word autocompletions.
This is getting ridiculous.
That does not mean every instance of offense should be ignored or dismissed, does it? How about addressing specific grievances instead of this extremely vague proclamation?
There are people up in arms about Maryland's flag because the checkered pattern is a confederacy symbol which somehow triggers them. The best course of action for those people is not to change the Maryland flag but rather to get them therapy so they can learn how to live without getting offended.
I can't give you an algorithm in advance that will let you decide every political and social issue ever. That's obviously not possible.
There are heuristics, like. You can put yourself in the other person's shoes, try to empathize with them to figure out where they're coming from, and work out what their values and their reasoning is. You're a human being, a social creature whose most distinctive characteristic is extremely complex and nuanced communication. Your brain was made for this.
> There are people up in arms about Maryland's flag because the checkered pattern is a confederacy symbol which somehow triggers them. The best course of action for those people is not to change the Maryland flag but rather to get them therapy so they can learn how to live without getting offended.
I'm gonna let you in on a secret. If you treat everyone with contempt and dismissal, then every conclusion you come to about them will be contemptful dismissal.
People often assume I'm a man on reddit. A lot of the time I'll politely let them know that not everyone on reddit is a man. Are you suggesting that I ought to go to therapy instead?
Its perfectly straight forward to write evocative and passionate prose without concreting to a gender. It makes sense that when we build general purpose tools we encourage them to choose the neutral words more readily because it increases their accuracy without any loss of meaning.
Are you really so terrified of a world where "meeting him" is "meeting them"?
Trying to remove bias from a system is about trying to remove bias from a system. People tend to agree that removing bias from a system rather than intentionally creating a system that discriminates.
If I'm writing an email about my investor I know who that is and what gender they are. Getting "them" as completion option is almost never what I would wan't in that case.
(Above I even used "they" subconsciously, because It was just an example and I don't actually have an investor to talk about here.)
ML driven automation widening wealth disparity? Eh.
ML ushering in the age of the persistent surveillance state? Eh.
ML being used to manipulate elections and hijack democracies? Eh.
ML might auto-suggest wrong gender in a text editor? STOP. EVERYTHING. THE RISK IS TOO HIGH IT MIGHT OFFEND SOME USERS.
1. People expect public organizations to not be bigoted.
2. People are generally fairly bigoted.
Therefore, if you're an organization training NNs on data produced by 2, you have to do some editing, or you're going to get in trouble over 1.
This would be similar to how feminist movements in languages that systematically distinguish men from women push to get the language changed so that women are referred to as men ("there shouldn't be any difference between a waiter and a waitress"), while feminist movements in languages that don't draw the distinction push to get the language changed so that women are referred to explicitly as women ("we should acknowledge that women can be waiters too").
If feminism demands one of these things, it surely can't also demand the other.
Different groups of people call themselves "feminists," and don't agree with each other about changes they'd like to see in the world.
It's kind of odd to me that people (like you apparently) don't understand.
I think they all agree that they'd like other people to change as an acknowledgement of their power.
I also think that they can't both use the same justification to argue for opposite changes. What did I miss?
"they" are different groups that might use the same label. They can absolutely use the same justification, to argue for opposite changes.
Their justification could be as simple as, "We want to be respected," and different groups of feminists disagree with each other about how best to achieve that goal.
Sure, it's even worse if you personally land in the crosshairs, but from your example it sounds like the feminists are working in different places with different languages. And perhaps the problem of gaining respect requires different solutions in different areas.
Including, apparently, the fact that some people are making an effort to care about other people.
The majority of people who "care" would have no cushy job otherwise.
This applies to programmers implementing useless features, social scientists, politicians, administrators and many more.
Assume there are 80% included in a game and 20% frozen out. Here it makes sense to change the rules of the game to include the other 20%.
Yes it might slightly inconvenience the 80% but the switch from not being included at all to playing the game for the 20% is huge. So, overall it might be a net positive.
Problem is when you get to a 98/2 split. Is it worth it to inconvenience such a large majority for the benefit of a very small minority. And how much are you going to change the rules?
Is it worth the reduced enjoyment/efficiency of the game?
At some point the trade off will become too much, and even moral/emotional arguments will not be enough to silence the dissent.
cf. https://medium.com/incerto/the-most-intolerant-wins-the-dict...
How much revenue would Google be losing over such a decision?
Even if something only affects 2% of the population, the cost of getting it wrong may not be worth the benefits.
Newspeak. When I first read 1984 I thought it so farfetched as to be laughable. I'm not laughing any more.
Please do not refer to madness as "your." "This" would
be preferable. Also, I do not recognize that "our"
language should change to address a tiny minority
wishing to be referred to in a non-binary sense.The analog between using computers and non-binary self-identification is a strawman/woman/person/thing. Which straw<something> will be acceptable?
Also the words in question are not used to address a person. When I am speaking to a person, regardless of their gender identity, I use the second person pronouns you, yours, or yourself. These are already gender neutral. The requested gender neutral pronouns (xir/xe/ne/???) are used in the third person and lower the value of communication as there is no asexual physical presentation, people present as men or women based on their fashion and physical characteristics. Going to a restaurant and saying to the attendant that you are here to me "that xir over there" does not help them figure out which table to take you to.
Not to mention that people who feel gender neutral are still male or female (or have experienced a physical developmental disorder, humans are dimorphic after all.)
Guessing something with a 50% probability of being wrong is not a useful feature.
> I'm pretty sure there are individuals claiming that NOT suggesting their pronoun deprives them of their identity, dignity or whatever.
Yeah but the end result of everyone wanting to feel special is that in the end, no one will.
One time I wrote a paper in high school where I didn't use any pronouns and always referred to Abraham Lincoln as "Abraham Lincoln", just so I could pad the paper as much as possible with the full length of Abraham Lincoln's name.
"Abraham Lincoln, in Abraham Lincoln's famous Gettysburg Address, suggested that Abraham Lincoln could..."
Nobody cares when touch keyboards predict the wrong pronouns, so why would the same service in an email client be any different?
A larger issue is that touch keyboards predict using single words and pretty quickly spiral into Markov chain incoherence if you let them draft whole messages. Smart Compose tries to offer syntactic and even semantic understanding. If I type "The investor is here, do you want to meet", my phone keyboard is working off 'meet', and will probably suggest 'up'. Smart Compose is working off 'investor' also, and apparently arrived at 'him'. (It's not clear to me from the article if this was actually tracked to specific biases, or if the predictions just adopted the default masculine.)
But most of all, I'm guessing Google's core motivation is explained by the offhand comment that Smart Reply uses the same logic. A gendered pronoun suggestion probably isn't going to spark a scandal, but a feature that encourages you to send Google-drafted text unedited might be subject to a lot more scrutiny.
Google doesn't actually care either. I hate call it virtue signaling but that's totally what it is even though that term is completely worn out at this point.
Err, so the AI correctly predicts that statistically you probably intend "him", but we limit the utility of the tool because that would be discriminatory? I know it will get it wrong sometimes, and you can say it re-inforces stereotypes, but if it will get it right most of the time based on strict statistical inference, seems like it could at least be configured. It seems to be a case where the AI is a bit too accurate... I don't disagree with their decision, I think given the circumstances it's actually a brilliantly safe move and keeps out of the fray as much as possible.
> “The only reliable technique we have is to be conservative,”
I know this is just whimsical, and a horrible logical equivocation on my part, but that's kind of funny that a large tech company decided being conservative might actually be useful in some cases to help protect against liberal outrage...
What axes are OK to use Bayesian inference on and what are not is a philosophical and historical question with lots of practical implications across politics, economics, actuarial science etc. It's really worth thinking about. But here's a start: in general, people are more forgiving of inferring data from mutable traits than immutable ones.
It's not something I would've thought to do myself, but I can see the reasoning for it. Bias in AI is a real issue, and it's wise to consider it earlier than later. Sometimes it's something more culturally visible, like in this situation, but oftentimes it can be much more subtle and insidious. This step isn't going to fix much but it's part of a bigger effort to make considering bias one of the priorities. Ignoring bias in our AI will make our AI more human in all the bad ways.
On the other hand, Smart Compose already seems like a bad idea to me. It's good for their AI but bad for humans. This pronoun action feels like a micro optimization for this technology's social impact, while the entire feature itself is a small net harm for society, in my opinion. It's a subtle dampening of our personal voice and nuances.
I mean, autocorrect can only take you so far anyway, your gonna have to change it a bit anyway and I assume most people with two brain cells knows the gender and preferred pronouns of the people mentioned in the convo and knows how to use them properly even if they don't have a clue about the pronoun debate. This is an edge case, but still something they saw value in avoiding a mistake in.
In more gendered languages, the problem is easier, since the AI could just look at nearby words to make a guess. In those languages, when it gets it wrong, people would likely assume that it just picked the wrong referent, instead of assuming gender.
https://www.wired.com/story/robot-gender-stereotypes/
> Robots don’t have genders—they’re metal and plastic and silicon, and filled with ones and zeroes ... The problem is that even if a robot isn’t gendered, and even if it doesn’t look human or even animal, you’ll tend to want to gender it.
However, because of Google's limitations, the language I use is modified. Their "AI" is shaping my use of language, and thus, how I communicate with others. Multiply this by millions (billions?) of people, and this could have a real impact on culture.
What is Google's responsibility, in this regard? Certainly they shouldn't ignore the ways their technology could affect society. They should be making conscious, deliberate decisions. This is dangerous territory; and not an easy one to navigate. I am glad they seem to recognize that.
When it's a machine doing it though, some unknown piece of metal spitting out a form message, it seems it would be easier to forgive.
I know this is annecdotal, but you did ask.
Edit:
In case, one is curious or unbelieving. Consider that my name is Nikita.
If Google really cared about combatting bias, they would just use “she/her.” At least it would be grammatically correct.
It’s not about taking a political/social stance to them, it’s about making life convenient for their users by removing any form of cognitive processing necessary
It's a tool that's meant to predict things, and they weren't able to successfully predict something that is used extremely frequently in lots of conversations. So they decided, after 'several' other attempts, that they were best off not trying to make suggestions.
If you can't do something right after repeated attempts, don't do it.
An incorrect pronoun isn't going to kill someone -- but we are gradually handing over more and more decisions to technologies that rely on AI/ML. Perhaps investigations into incorrect pronoun assumptions can lead to improvements in assumption errors in other areas (e.g. you think your self driving vehicle doesn't need radars and that only using cameras is sufficient? Just because it looks like a big fluffy cloud doesn't necessarily mean you can safely fly/drive through it).
Back to just pronouns though: if someone says "My teacher assigned me to read Act I of Macbeth tonight", most people would avoid a reply that uses he/she until an indication was given or they'd just ask "Who's that, Ms. McFadden? Yeah, I know, she assigns way too much homework!". If a human can be smart enough to get it right, then I'm glad folks are working on AIs getting it right too (or, for now, not making assumptions until they can get it right).
The fact that AIs are advanced enough for us to be thinking about these kinds of details is wonderful! :)
Yes I realize that saying this on hacker news or reddit is grounds for heavy downvotes and hate mail but good god people. This is all political gaming, not compassion.
Here you go Google, I fixed it for you
These times are indeed hyper-sensitive, and ridiculous exaggerations do occur. This is not one of them.
Gender dysphoria is a real, crippling, dangerous mental illness. Maybe in the future there'll be some pill or simple brain surgery that fixes it. Today, there's none of that.
Gender reassignment surgery generally helps. Calling people their preferred gender helps, as a complement or substitute to surgery. It's a hacky, inelegant solution, perhaps disgusting to some, but it fucking works! Goind the extra mile to avoid misgendering people is reasonable.
"Imagine if we could give depressed people a much higher quality of life merely by giving them cheap natural hormones. I don’t think there’s a psychiatrist in the world who wouldn’t celebrate that as one of the biggest mental health advances in a generation. Imagine if we could ameliorate schizophrenia with one safe simple surgery, just snip snip you’re not schizophrenic anymore. Pretty sure that would win all of the Nobel prizes. Imagine that we could make a serious dent in bipolar disorder just by calling people different pronouns. I’m pretty sure the entire mental health field would join together in bludgeoning anybody who refused to do that. We would bludgeon them over the head with big books about the side effects of lithium."
From http://slatestarcodex.com/2014/11/21/the-categories-were-mad...