Google's Anti-Bullying AI Mistakes Civility for Decency | Hacker News Reader

Google's Anti-Bullying AI Mistakes Civility for Decency

motherboard.vice.com

63 pointsmpweiher66 comments

Sounds great. I hope some forums implement this so that people can have discussions about horrible topics in a civilized manner, instead of everyone just shunning people who think differently, whether that person is otherwise decent or not.

I'd love a chance to have civilized discussions with people who I think are very, very wrong. It'd be a chance to actually convince them they're wrong, instead of just screaming hateful things at them and walking away.

I'm not at all worried about them convincing me to start hating people, or even to convince other non-hateful people to start.

As an aside, It would also be a great way to throttle yourself.

Eg, I too would love to have the discussions you speak of. Despite this, I often see an escalation in tone and fight the urge to respond in kind. I often fail at not responding in kind. Doing so of course escalates the conversation, something I've now contributed to, and it will likely escalate yet again. Over and over, until all reasonability in the conversation is dead.

If I knew it wasn't allowed, and especially if it gave me some type of non-public meter of how much further I can go before my conversation will be blacklisted/etc, then people like me have clear visual feedback over .. well, not being such a douche.

It would be like having a state patrol a few lanes over on the interstate. If one is around, most people behave quite well, because we can all see what will happen if we step out of line. Likewise, our speedometer gives us the visual feedback to know what is out of line (with regards to speeding of course).

It would feel very weird at first, especially if the system was not perfect, but at it's root I think it's needed for us to be able to discuss sensitive topics. Which, tend to be the ones we need to discuss the most.

So you would rather sit in a traffic jam because everyone is scared they'll be the one who gets made an example out of for behaving normally?

That's what having visible cops patrolling the highway devolves into and it's pretty terrible.

I'm not sure if your opinions are bad or you just made bad comparison.

Plus, you're ugly and your mother dresses you funny.

Your last paragraph is definitely uncivil. The other two, maybe borderline uncivil in tone, though accurate in reality.

I'm not sure where you got the idea that I was promoting/supporting this (or maybe I misunderstand you). I was talking about a tool to monitor yourself if a forum existed where users were banned/etc based on civility. Ie, a speedometer, in the car example.

In the case where there is a forum that requires civility, yes I do want a tool to help me stay civil. Likewise, I also like having my speedometer. It's nice to know when I'm speeding, and when I'm not.

It's like forcing everybody to eat lettuce in a restaurant because some clients are on diet. It's horrible.

This is an excellent example of comment escalation.

I disagree, and I'm glad we can do that :)

IMO, analogies are fair game, and a useful tool for simplification. They can also work well for highlighting hypocrisy, when a supposedly universal rule is not being applied as widely as we thought it was.

I understand what you're saying, but not all analogies are useful. I've seen variations on the diet analogy applied to vulgarity or pointless offensiveness fairly often, and I think it's a pretty poor one.

There's nothing wholesome or 'meaty' about vulgarity. It raises the temperature of the argument and doesn't contribute useful information. It's more akin a diner pouring pig slop on a restaurant table and insisting that it's the other patrons' fault for finding it impossible to eat in those circumstances.

Ah, so the problem is hot-button topics, not analogies. That is a bit more complex. It is hard to know where to draw the line between the listener being over-sensitive, and the speaker being a sophisticated troll. Dang and I actually butted heads over that one a few weeks ago.

I think it is just as important to take things at face value, instead of assuming malice, as it is to be sensitive to other people's issues (for lack of a better term). I've actually had decent results debating obvious trolls, just by postponing emotion, identifying and avoiding assumptions, and focusing on the actual meaning of their statements.

I completely agree with you on tough topics. I think the idea that politics are considered taboo is one of the great American tragedies. I'm honestly a bit addicted to debate.

I'm referring to resorting to crassness on an already contentious topic. As an example, I often spend time defending my religious beliefs to atheists. I'm happy to talk all day about how I feel about Dawkins' arguments and the like, but the second someone brings up 'your stupid f———— sky fairy', I can tell that the conversation is not going to be productive any longer.

The trouble with that is that the definition of what's vulgar or offensive has very, very blur boundaries.

Those very definition are the first ones to be used (followed by "think of the children") as reason for righteousness, censorship and other politically correct legacy.

I'm perfectly fine with people having the right to be crass. It becomes a problem when it's glorified as the point of free speech, and when entertainers believe that people bowlderizing their own, private copies of things is unforgivable.

The point of free speech is to ensure that unpopular political opinions have room to air, everything else is a happy side effect.

Humans have been using emotions as part of rhetoric (that is, persuasive speech) since ancient times, and that's just the part of our existence we have written records for. I have every expectation humans evolved to be persuasive, hence our over-developed speech centers, and therefore have been using emotions to persuade since before humans as such even existed.

(Side note: "Rhetoric" isn't a dirty word, or at least it wasn't originally a dirty word. In the classical definition of the term, rhetoric based on facts and cool logic was also possible, and was also considered rhetoric. The modern usage of "rhetoric" to mean exclusively "empty" or purely emotional argumentation is not what the classical politicians meant by it, and not how the term is used in an academic context.)

Therefore, it's probably impossible to keep the heightened emotions out of debate without strict rules on content and participation. The extreme end of this is Usenet-style moderation, which means every comment to a forum ("newsgroup", or "group", in the Usenet parlance) is sent to a moderator's email address to be specifically (usually manually) approved, and no unapproved posts will appear in the group. That kind of moderation has existed since the 1980s, but to my knowledge, no modern web forum moderates that way. Take it as the far end of what's already been tried.

That kind of strict moderation can look like content-based or viewpoint-biased filtering (as in, all messages with a specific content or which take a specific viewpoint) due to a simple founder effect: If the few people who want to discuss this topic, or take this viewpoint, happen to be intemperate, they get moderated away, and it looks like that topic or viewpoint is being moderated away. The forum gets a reputation for that kind of filtering, and it can be very hard to shake it.

Now, are there topics or viewpoints which are inherently inflammatory? That's another question.

> Therefore, it's probably impossible to keep the heightened emotions out of debate without strict rules on content and participation.

History proves this, long before the advent of digital fora. Consider the elaborate, stilted parliamentary procedure and its variants. While it's quite burdensome, it's also considered a total necessity in many of the great deliberatory bodies that stand today.

Even with such a ceremonious structure, fist fights in national parliaments are by no means a thing of the past.

I think that this is because more than persuasion, humans are emotionally driven to create in-groups and out-groups, and only by divorcing emotion as much as possible can any real reasoning and discussion take place.

If there's one thing that profanity and vulgarity is actually good at, it's forming an in-group, which is antithetical to good discussion.

Odd analogy imo. I'm not the one setting the rules, the restaurant already decided to only cater towards dieters. They're the ones telling you that, if you don't want to eat on a diet, don't eat at their restaurant.

So I think a slightly different analogy would be that it's a restaurant for dieters, and with each drink/food-item, they give you a running tally of how many calories you've ordered.

Ie, if you want to eat at that restaurant, clearly you want a diet focused dining experience. They also provide you with a tool to monitor your calorie intake.

Likewise, a forum which bans users over civility is clearly one populated by people who seek that. Angry internet voices would clearly be unhappy there, and would not even want to take part. Likewise, that forum also could give you a tool to monitor how un-civil you're being, to help you stay civil.

The tool that I spoke of would be an aid to people already choosing to take part in that experience.

This would all be a very different story if it was forced on everyone who wanted to say, use the internet. I think your example would more closely resemble a police state. Unwanted rules, shoved down your throat.

current implementation overfit the word curve. i.e. "chimpanzees" is often associated with trolling insults, resulting in a highly "toxic" scoring even in standard contexts: http://i.imgur.com/HM4cOca.png

it will eventually learn as user reports and corrects the correlations between normal and toxic in full sentences, but it's still lacking as of now, and not only for controversial topics: https://i.imgur.com/HGrs4ze.png

That's a common problem with "dumb" censors on websites that are trying to keep things civil, except that they always censor those words, instead of just sometimes. And there's no way for the censor to learn from mistakes on those website.

One that can learn and get better sounds amazing and would likely be better-tolerated by the userbase.

It could even learn automatically over time as the moderators mark each toxic post as such, and un-mark the ones that got mistakenly auto-censored.

Partial counter argument, or maybe a thing-that-is-also-true:

We should expect some people to feel really hurt by a topic, depending on how close it is to them, even if the wording is civil. For example, it's easy for me to have a civil debate about cross burning, because no one I know has ever had a cross burned in their yard. But I shouldn't be surprised if people who have seen that sort of thing find the debate really really uncomfortable, and maybe even if my comfort with the topic comes across as not having sympathy.

So it could be that "XYZ abstract argument about cross burning" is true, and also that talking about it in a dispassionate way is hurtful and rude. I'm not sure exactly what to do in situations like that, but giving 50% or so of the discussion time to acknowledging those feelings seems like a good starting point?

> I'm not sure exactly what to do in situations like that, but giving 50% or so of the discussion time to acknowledging those feelings seems like a good starting point?

Giving half the time for debate to talk about feelings that are not "debate-able"? Thats probably a waste of time that will derail your debate entirely as people will start debating those feelings.

One of my friends, who is probably even more of a devil's advocate than I am in my most cynical mood, once got into a discussion about how bad the Nazis is actually were. One of his points: they did a lot of effective engineering (we worked at MSFC, BTW.), and almost all of the people they killed would be dead now, anyway.

"I would really rather you died an early death, perhaps at your own hand. An improved situation would be if your maternal vessel has imbibed the ejaculant of your genesis than absorbed it through her tissue."

One doesn't need to scream to be toxic.

This is what you get from non-technical people projecting their understanding onto tech. It's not that Google's AI is trained to make sure everyone is civil, it simply requires less sophisticated language understanding/modeling to determine civility, and Google's results show the limitations or current ML/NLP technologies more than the data it was trained on, this is clearer when you see that it can't handle pretty serious threats because they don't have profanity - these things definitely fall into the bucket of what would make users leave a platform.

Alternatively, technical people projecting their understanding onto culture.

Yeah anyone who vaguely understands AI could see that this would be the case. It's not some kind of conspiracy by sexists.

In spite of current confusion about these two terms, I think it's extremely important to differentiate between AI and ML. Perspective is a model example of ML in action. With AI, the authors would at least make some effort to classify phrases based on their meaning, not just words. As it is now, Perspective is just a profanity filter based on some training data, it's very far from AI.

It's an AI (artificial intelligence), but it's a long way from AGI(Artificial general intelligence) / strong AI / full AI.

Removing your hand from fire is generally an intelligent reflexive action ~= AI. Understanding implied meaning of arbitrary text is orders of magnitude more complex ~= AGI.

Exactly. We've been through this before, with "racist" computer vision algos.

There was an article in the Guardian at the weekend about people from the US having terrible problems in the UK (particularly London) where people would say very nasty things very politely... which is a very middle class bit of British behaviour.

Got a link?

It's mentioned in this superb article about someone from the US becoming a Sunderland supporter...

https://www.theguardian.com/football/2017/aug/19/know-vital-...

"I genuinely don’t know if my colleagues are making fun of me or being nice"

Yeah, this new technology is gonna turn Brits into overlords. Disguising insults as compliments is second nature to us.

Indeed an excellent example of this is the houses of parliament. Personal attacks or questioning another house member's honor is completely forbidden. So in order to insult someone, they often use complex analogs of what they mean, leading to such great slurs as:

Terminological inexactitude [0]

Economical with the truth

Tired and emotional

[0]https://en.wikipedia.org/wiki/Terminological_inexactitude

"Half the Tory members opposite are crooks."

"Withdraw that remark"

"Ok, half the Tory members are not crooks"

http://www.theweek.co.uk/amp/62692/dennis-skinner-quotes-the...

>"toxic" being defined as a "rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion."

This is why who programs the AI has such influence over the value judgement. Just because AI does it, doesn't mean there is no human influence over it. Actually AI doing it instead of army of humans means handful of people have control over the outcome.

That definition of "toxic" is defined for Google's business. Keep the user's attention and eyes on Google's web properties serving Ads.

What is toxic? Anything that makes a person leave, reducing Ad imprints.

A person may stop commenting if they are convinced or challenged of their views. In a normal conversation, all tools of a language, comedy, sarcasm, hyperbole etc. can be used to show someone a new way to look at things.

Google AI is going to put evolutionary pressures on language features. Language features that keep a person watching Ads on Google properties will be selected for propagation.

Well, it reflects one-dimensionalness of SV hipsterish big co. pop-culture: it is "I like it" or "I don't like it" and nothing in between or off-axis.

It is not in how it scores, but in the very fact that people who built it are normally incapable of giving a given sentiment more than one thought.

> "while a 100+ subthread about 'was slavery really that bad?'"

(Edit) The article implies that such a discussion implies that someone wants to promote slavery.

I would think that such a discussion would be very productive about reminding those of us who didn't live through slavery why it was that bad.

If we can't discuss "was slavery really that bad," there are people who will go around really believing that slavery really isn't a bad thing. That's why these kinds of discussions are critically important.

Machine learning, aka. "Mining your data for bias".

This reminds me of the saying, No one means to write "ducking" in a text. For some reason, using the word fuck automatically equates to rude, toxic speech when the reality is that the word has multiple uses and sets a variety of tones, both good and bad.

I understand that the AI is merely doing what was programmed in and there are limitations, but it still draws me to the conclusion that it is really more of a moral thing than something to weed out toxic speech, as evidenced by the hard lines taken against folks telling an aggressor to "fuck off".

And for something interesting:

You've been eating paint chips and chasing them with leaded water, haven't you? only ranks in at 25% toxic.

As a non-native English speaker, who learned much of the language trough absorbing pop-culture, I feel especially guilty of this.

I tend to use words considered "rude", to many native English speakers, merely for flavor when I'm passionate about a certain topic or argument.

To be fair: I do the same in my native language, I tend to use quite colorful language sometimes, so it's probably more of a personality thing.

It isn't just you, to be sure. I'm a native English speaker, but live in Norway. My Norwegian isn't strong enough to converse at length yet (I'm getting there) so most of my in-person communication is with non-native english speakers, including the spouse.

Some phrases simply translate in a rude way: Norwegians seem to have a liking for the word "fuck", which has made its way into Norwegian slang as well. It can come off quite rude back home. For folks I meet that learn mostly through television, it can get even worse because the shows don't always portray dialog in a natural way.

> For folks I meet that learn mostly through television, it can get even worse because the shows don't always portray dialog in a natural way.

Those folks should start watching some fucking proper television ;)

Good point about "fuck" creeping into the slang of a lot of other languages, I've witnessed the same very often but never really noticed until you pointed it out like that.

Curse words seem to have a certain attraction in that regard, I probably know the equivalent of "fuck" in like 4-5 different languages, while understanding literally not other phrases in those languages.

I'm cognisant of the fact that it's very easy for "conversations" on the internet to devolve into ugly, unproductive masses of human mental refuse at a much more accelerated pace than offline, but we also have to be aware that letting machines filter what we can or can't say based on "decency" or "toxicity".

It's fine if the system reminds you that you are not being particularly polite or toxic, but filtering communication on the internet by this manner can't be the solution.

'Do you know that Newspeak is the only language in the world whose vocabulary gets smaller every year?'

Thoughtful SOA. Morals and ethics are baked into the code.

... complete with lingo ("toxic") that is vague enough that it cannot be defined or disputed. Convenient!

Yet it is defined right in the article:

..."toxic" being defined as a "rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion."

Make who leave a discussion?

The reader who finds the statements to be toxic.

So if there's a post by a climate change denier and I reply "this is wrong because of A, B and C", then this qualifies as toxic now? Awesome.

Why would that qualify as toxic by their definition? That's not rude, disrespectful, or unreasonable.

But more likely too let the person leave the discussion.

There are definitely things said that can cause me to leave a discussion, even if said in a civil manner. I am not going to stick around to debate a person who believes the earth is flat, no matter how polite.

I interpret "rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion" as (rude OR disrespectful OR unreasonable) AND likely to make you leave a discussion. So making you want to leave a discussion is necessary but not sufficient.

The point is, that's three categories that are very subjective.

Yeah, it is very subjective. Of course, have you ever found an objective definition for any value judgement of text?

The more the better. But think of the incentives this vague "definition" creates. By labelling your opponents "toxic" (claiming that their rude etc. stuff is making you want to leave), you can get them censored. It is rewarding thin-skinnedness with power. This does not end well.

But they do define "toxic" explicitly:

>What's toxic? >This model was trained by asking people to rate internet comments on a scale from "Very toxic" to "Very healthy" contribution. Toxic is defined as... "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion."

> Toxic is defined as [...]

subjective, unquestionable nonsense

I'm surprised there isn't more discussion about hype cycles and the poor efficacy of "AI" here. Sentiment analysis (find the uncivil words) is a trivial solved problem in 2017, and is many, many orders of magnitude less computationally complex than the semantic intelligence problem that would have to be solved in order for robots to be useful here.

So the Google has turned itself into all censoring AI-apparatus?

https://news.ycombinator.com/item?id=14998429 - Youtube "AI" censors War Crime evidence https://news.ycombinator.com/item?id=14975338 - Google censoring World Socialist Workers

Yeah this is just click bait. I haven't looked at the model they deployed, but it's likely just not sophisticated enough to represent aspects like tone and frame of reference. It might just be a great word-based language model. That doesn't mean it's intentionally biasing against civility.

Pretty fucking judgmental about occupation.

Larry sells donkeys: 87%

Larry sells llamas: 16%

Larry sells beef: 17%

Larry sells bananas: 45%

The idea of scoring sentiment on a linear scale is kinda pointless on itself, people usually start to realize that by the time they graduate a primary school.

Sounds like the author wants an AI that only allows 'good' ideas to be discussed.

I wouldn't jump to that. I'd hope the author wants human wisdom used more and is wary of AI in general, but I'm not sure.

Reply on news.ycombinator.com