Instagram launches hate-speech 'limits' to hide abuse | Hacker News Reader

Instagram launches hate-speech 'limits' to hide abuse

bbc.co.uk

46 pointsFridayoLeary86 comments

With how social media turned out to be, is there any sustainable benefit for the users in participating?

Users locked themselves into a system that provides abusers and other trolls a very direct access endpoint into the user‘s emotion. If you receive suddenly 10.000 intimidating messages - even if Instagram now hides them automatically one click away - you still will feel very agitated and it might ruin you day, night or even life substantially. Mind, that there is a substantial imbalance which results in a situation of duress for the user: The user gardened their profile in the social network over a very long time, decades even, on arguably more than a daily basis and will perceive very high cost if they remove themselves from the social network. Meanwhile the abusers can create new accounts and doxx the users with virtually no cost.

Users locked themselves into a system that siphons out any personal information they willingly and unwillingly give up and which is used directly to manipulate or trick the user into changing their behavior towards whatever the social media wants.

Users locked themselves into a system that fosters a show off culture and spreads a fake perception of reality and feelings of envy, inferiority and vanity. Continued and sustained usage leads to dependency and depression.

Not even to talk about how the culture of political discussion and bubble of misinformation harmed our democracy.

What do the users get from this? What do we get from this? Why should anybody use social media?

Why shouldn’t we see social media as a public health issue?

I am using social media to be - social. My facebook friends are mostly composed of people I have interacted with in real life first.

I use Instagram/Facebook to display the photos I want to publicly share, easiest way to get some audience, especially people I know.

I use Twitter mostly as a news service to follow people whom I know to post interesting content. Especially Twitter can be a valuable source of very up to date information if you have a trending hash tag. E.g. with the recent flood disaster in Germany, you could get a stream of posts from the affected areas.

The trending hash tag can though be quite a snake pit, as, depending on the tag, a lot of bad posts can appear too, there are a lot of hash tags to avoid :)

That isn't how life is supposed to be, you're manipulating your image and selling a version of yourself that isn't truly you.

Twitter is also not news.

Sorry, how you state it, it is just plainly wrong. I am not manipulating my image at all. I am not posting images of my self, but general natural photography. Animals, flowers, landscapes. I am not selling anything.

I also have not claimed that Twitter would be "the news". But it is a great source of specific information. To repeat the example I had put into my previous post: when a disaster strikes a certain place, the best chance to find immediate information like pictures/videos taken is people affected tweeting. This is raw material, just some data. This is of course to be complemented by the news media, though these often enough are limited to the same sources, but they might do some aggregating and processing. Often enough news events never make it to the news, because they are only of local interest.

Also Twitter is often the outlet of official organizations. Like the police of a large city informing the inhabitants about events they should be informed upon.

> Also Twitter is often the outlet of official organizations. Like the police of a large city informing the inhabitants about events they should be informed upon.

That's definitely the case where I live. It's second on my go-to list after our local news website.

Pre-social media vs. Post-social media:

Pre-social: Find people you have things in common, hang out with them and enjoy life.

Post-social: Find people with which you only have disagreements, harass them, ridicule them and live in misery because they exist.

As with Blizzard, Facebook seem to have some deep aversion to manual moderation, and will spend millions on getting developers to make half-hearted attempts to automate it without actually solving the problem.

This is a real band aid that will do nothing to dissuade people from continuing their abusive behaviour.

Manual moderation scales linearly: for X posts per day, you need nX moderators, forever. Silicon Valley execs are allergic to anything that doesn't scale sub-linearly. It makes them feel like the singularity might not be around the corner.

>will spend millions on getting developers to make half-hearted attempts

What do you think the annual cost of manual moderation would be?

Facebook can afford manual moderation. What do you think the mental health cost of waiting for AI moderation to be good enough in 111 languages that understand the nuances of speech in text/audio/video?

I'm not saying that the expense of manual moderation would excuse doing a poor job of moderating, but the top comment talked about spending "millions" on trying to automate moderation as if manual moderation wouldn't cost as much. They're certainly spending many millions on both manual and automatic moderation as it currently stands.

They've spent tens of millions of dollars just settling with the human moderators for their job-related mental health issues https://www.theverge.com/2020/5/12/21255870/facebook-content... .

Manual moderation doesn't scale. It needs to be automated.

If scales just fine. It's embarrassingly parallel even.

It's not a scaling problem - it's a provisioning problem. And Facebook isn't willing to pay for it.

Manuel moderation is not the solution. The issue is FB (insta) is the one moderating. Give moderation powers to the users, create a market for moderation. Some people will want whitelists, others AI screening models, others “manual moderation”. Let users decide what they want.

YouTube tried that and people found ways to abuse it immediately. Organized groups started going around getting chennels they don't like banned on false pretenses.

YouTube had differentiated moderation for each user based on personal preferences set by the user? I don’t recall that.

No, they tried just one of the methods you listed. Having it be set by user preference is an interesting idea, but whatever the default setting is will be used by the overwhelming majority of users and will be quickly subverted by attackers.

Gotcha. I agree any default setting needs to expect attackers. I’d set the default to whitelist content from Insta (so they manually approve it) and verified real people “friends” can white list additional content for you.

If you want to open it up more you can and layer in content moderators for hire, AI models to try, dashboards to optimize what you want, etc.

The GP was probably suggesting letting bubbles of people moderate for those who opt in to the moderation. Thus, a channel would only be banned for people who are in the group.

Analogously to real life, one doesn't have the option to choose all of ones neighbors, but you can choose the neighborhood you live in (assuming you have adequate wealth, redlining is illegal, etc). Moderation bubbles would enforce content rules much like an HOA enforces grass cutting, etc. I'm not sure how it would work out, but I'm curious if there are existing examples.

Is it a right to post as many times as one wants or can we consider a throttling mechanism?

Obviously throttling means less revenue and that collides with commercial interests. However, given people are over reliant on social media and social media with engagement, moderating this perniciousness might be worth trying out.

> This is a real band aid that will do nothing to dissuade people from continuing their abusive behaviour.

Band aid, yes.

The only thing that will change behaviour is consequences. I don't verbally abuse people face to face because there is a non zero chance of being punched. I am almost entirely liberated from consequences in the online world.

Online I can't see the person I'm abusing, I don't see their pain, I have zero chance of empathising with them.

Even verified accounts wont help, if there are no consequences to me abusing someone. The next best thing is to hellban. (in my opinion.)

The problem is, doing that at scale is exceptionally hard.

> I don't verbally abuse people face to face because there is a non zero chance of being punched. I am almost entirely liberated from consequences in the online world. Online I can't see the person I'm abusing, I don't see their pain, I have zero chance of empathising with them.

This makes me think that if I am physically weaker and guaranteed not to beat you, you will abuse me in person until I cry or am otherwise seriously disturbed.

That's the point. KaiserPro probably won't do it, but a lot of people certainly will. Online, however, it doesn't matter how strong you are. Anyone can be a bully or be on the receiving end.

Furthermore, if "the other side" on any particular issue is spewing vitriol and harassing people then your side loses a little by not doing the same because the other side's behavior will (at the margins) cause the normal people caught in the middle who might support you to keep their heads down thereby making it look like your side is less accepted and their side is more accepted. So not only will people do it but it is incentivized from a game theory perspective.

I should have written this as a third person. I was trying to get across for people who think empathy is weakness, that we all moderate our behaviour because of consequences.

I don't abuse people in real life because I have a deep seated need to be liked. Pissing people off makes them not like me. This is a consequence.

Seeing someone disapprove of what I'm saying, as I say it, burns.

I don't verbally abuse people face to face because there is a non zero chance of being punched

Just so we're clear...if you live in the US, you absolutely have a right to stand on the street corner and abuse me as long as you're not likely to cause imminent lawless action...all of that without getting punched.

You absolutely can call for all people of my ethnic/racial group to be deported(or worse). What you can't do is call for your followers to go kill me or burn down my house or any other lawless action.

You DO NOT have a right to say those things on facebook/youtube.

In case anyone was thinking of bring up the "fighting words exception", please read this https://www.thefire.org/misconceptions-about-the-fighting-wo...

He wasn't talking about legal rights, he was talking about real consequences.

exactly this, I'm not talking about legal consequences.

I'm talking about literally being punched in the face for saying something that you'd hear a 14 year old gamer shout over the microphone. In the real world, they'd never say it to my face, because they know I might tell their parents

> have zero chance of empathising with them.

Wow. At least you can admit this. To me, this summarizes most of the divisive nature in today's society. People just cannot empathise with other people any more. It's a very sad bit of social commentary. To me, rather than bringing people together, social media has lost that bit of humanity where people seem to just think of other users as non-people and just some anonymous chatbot level of emmotionless rather than an actual human.

We should develop a method so that when you make an online comment that you would not make in person for fear of being punched, the readers of the asshole comment can trigger your monitor to release a punch. It can be as outlandish and cartoonish as Data's boxing glove on a spring from the Goonies. So now, instead of just being afraid of being punched by the one person the comment was made, you must now be afraid of being punched by anyone on the internet that can read the comment.

For those too young to have seen the Goonies, jump to 1:35 for the reference: https://www.youtube.com/watch?v=MLzupaRrzAY

Empathy is a learned skill, one that we don't teach.

in the online world you have to chose to empathise, and its really easy not to. This is why I seek out what "the other side" are thinking, and why they say what they do. Because otherwise its really easy just to dismiss them as "stooges", "snowflakes" "nazi" or any other signposted pejorative.

but I'm not perfect, as my comments will show. Sometimes I fail.

>Empathy is a learned skill, one that we don't teach.

one that we don't teach, anymore. maybe i'll go along with that, but it was definitely taught when i was a kid. "how would that make you feel if they did that to you?" was a very common chastisement growing up. not just from my mom, but from teachers in elementary school as well. then of course being drug to church by my mom, there was always people spouting "remember the golden rule". which when it comes down to it, is the only thing i really "took" from church. i treat people how i would like to be treated myself.

so, if none of that basic stuff is being taught nowadays, i weep for humanity ;-)

btw, we all fail, we just have to strive to do better!?!

The aversion is due to cost. Simple as that. Manual moderation is simply too expensive.

There are literally not enough humans in the global workforce to moderate all the content that goes through Facebook's properties. Instagram alone has over 20k things posted per second.

Facebook processes literally hundreds of millions of posts per day. The idea that it can be human-moderated is laughable.

Not an endorsement, but you might want to take a look at the scale that human moderation is currently applied at in China.

So we should involve no humans at all? I think many people are clamoring for something instead of the status quo of nothing.

If the alternative is doing nothing, then maybe we need to change the legal framework to make businesses like Facebook unsustainable.

Why should the size of a business be an excuse for that business to operate antisocially? We don't say "Dow chemical processes millions of gallons of waste fluid every day. The idea that they can avoid dumping most of it in a river is laughable".

If Facebook can't profitably operate their business in a way that is acceptable to the society they operate within, they don't have some inherent right to a profit.

I didn't say it was acceptable. I hope they all implode.

It can be. There is a button to flag posts. Moderation can prioritize by the flagging and the amount of flags a post gets. Considering how many analysis is done onto the facebook users, it should be relatively easy to come up with a scoring system. Could be as simply as how often a post by a certain user is flagged (and potentially removed by moderators). Also just randomly check posts can be effective, if consequential action is taken.

Yes, this is some effort and will cost money. But we are talking about hugely large and profitable companies here.

While it might not be possible to "solve" the issue entirely, it could get a lot better if done thoroughly.

They already do everything you suggest. Clearly not as well as people would like, but this is like Moderation 101. They get better all the time at automatically analyzing things to have humans intervene. It still sucks, they will never be able to keep up. People are monsters and don't deserve nice things.

Everything is a bandaid when you're policing what people say. Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done.

Moderation on social media isn't something that can be solved satisfactorily, everyone is always unhappy with it. Take the lab leak theory with COVID, originally Facebook came under fire for allowing misinformation to spread so was blocking content which contained this claim because it was considered harmful but they reversed course once traditional media such as Vanity Fair began reporting it. Regardless of where you stand on this, you're unhappy with what they did.

Even hate speech is pretty ambiguous. The issue is one of subjective interpretation. Some comments are unambiguously hateful, others are hateful to some and not to others due to their cultural context which is important when you consider that these are global platforms. I've found it much easier to not use them than worry about what's on them.

> Manual moderation is unlikely to solve the issue to people's liking either, it doesn't on here or anywhere else I see it done.

I wholeheartedly disagree. I think the human-approach of HN mods is _the_ reason this site is better and different from other social/forum sites.

HN is a fairly ideologically homogeneous community. Moderating it is "easy mode" compared to FB or somewhere else that has a broader cross section of the population.

The point of moderation is not to make everyone happy. It’s to maximize profit, which is a second order effect of the size and engagement of the user base. If assholes are being toxic on your platform, that might be good for a while — until people start leaving and the media starts reporting on it.

If someone is being an asshole to other patrons in a bar, they get thrown out. No one worries about whether they’re happy with the decision. No one says “actually their comments are only subjectively asshole-ish.” We make these judgments constantly, in basically every arena in life. Why do we start wringing our hands when it comes to online platforms?

I don't know if this is true for everyone, but I had no issue a decade ago. I only started wringing my hands once I started seeing people abuse online moderation policies for political gain. I've seen increasing evidence of that over the years, and the big crossover point for me was in October, when Twitter moderators banned a newspaper article about a presidential candidate's son under the premise that obtaining his private documents was hacking.

It would be a good start if any clear hate speech and very personal attacks would be moderated and prosecuted regularly.

The manual moderation here is pretty terrific. No doubt there are some that feel abused by it, but I'm sure they can find another platform more to their liking.

How many people do you think it would take to meaningfully apply manual moderation at the scale of Blizzard, Facebook, or Instagram? How many more people in support staff? How much do you think this would cost?

How well do you think that cost would stack up against spending millions on half-hearted automated moderation?

The real problem is that no company is qualified to be the moral arbiter of all society. All such attempts are doomed to fail as the only success will be the furtherance of our dystopia.

They don't have to be the moral arbiter of all society, just moderate their own users. The problem is that they want all society to become their users. They are biting off more than they can chew, to the detriment of civil society.

>The problem is that they want all society to become their users.

That's not a problem. Even if all society became Instagram users, Instagram couldn't act as the moral arbiters of all society, they can still only moderate their own users on their own platform.

Substitute that with any platform, including Facebook and its billions of followers. Facebook isn't society. Facebook hasn't replaced or supplanted governments, doesn't control the media, doesn't burn books or dissidents, doesn't censor any of the countless alternative means of communication that exist. No matter how big Facebook gets, Facebook can only control Facebook.

Even Facebook itself doesn't have this viewpoint. They are openly acknowledging they want to influence behavior in the real world outside of the mere use of just the Facebook app.

I don't know what specifically you're referring to. A lot of companies make bold claims about wanting to "change the world" and whatnot. I'm not aware of any public Facebook policy where they acknowledge wanting to take over all governments, control all thought and communication, and act as the sole arbiters of truth and morality for society.

I hardly think Facebook has a deep aversion to manual moderation, considering that I've read they employ around 15,000 manual moderators.[1] However, you may still be right that this is the wrong approach in this case.

[1] https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-...

Can we just admit that making social media a "global town square" has been a horrific mistake, and just move on?

Gross oversimplifications like these that are barely related to the specifics of the article somewhat make your point

nice

This is a terrible way of handling it. Hiding comments? Flag abuse and ban people.

Why is that so hard?

Because they make money from outrage.

You're putting the onus on the individual to identify and flag abuse, meaning they still have to experience it.

If you're trying to say that automatically flagging all abuse correctly isn't hard then I don't think you appreciate the problem.

Seems logical. long time followers will be more levelheaded on average

I had my first experience with it today. I used a yiddishism in a comment on a post about one of yiddish institutions and was greeted with a hate speech pop-up. IG is definitely going to purge quite a lot of colloquialisms with it.

It's probably the same sort of dumb bag-of-words classifier that made Google's toxic speech identifier classify sentences like "I am gay" as "extremely toxic".

You give each word a score and sum them together, completely ignoring context and meaning.

People from Spanish speaking countries have had their FB accounts limited for asking about black ink cartridges for HP printers or black HP printers in computer supply pages.

https://larepublica.pe/tecnologia/2020/06/14/facebook-volvio...

https://www.fayerwayer.com/2020/09/facebook-imagen-maldita-b...

That's appallingly ango-centric.

So weird, why would a yiddishism have a negative speech score?

Edit: realizing this comes across as sarcastically anti-Semitic. This is a genuine question, I don't understand why a yiddishism would carry such connotations in the automated hate speech system.

In the context of the last trend of nlp a word is defined by its context, if a word is mostly used as a derogatory term in the corpus that the model has been trained it will most likely be used as a flag to classify the content as hatespeech

Do people use yiddishisms derogatorily?

Not sure if I’m understanding “yiddishism” correctly here, but internet anti-Semites do mock kvetching all the time.

Yes, since internet anti-Semites mock kvetching anyone using yiddish words in a comment on a post about yiddish institution is an anti-Semite engaging in hate speech.

https://www.yiddishbookcenter.org/collections/oral-histories...

I'm not sure your point?

Absent the original commenter (you) providing context, that makes for reasonable supposition. Automated systems learn from a corpus that includes internet anti-Semites using Yiddishisms in a mocking manner to insult and inflame leads to false-positives on real-world, genuine uses.

Similarly, how does one fish out the context and gauge just how appropriate (according to HN, never) a use of "n----r" and all its variants? Consider whether one should tell Joe Schmoe Black American he cannot use the word all to prevent Joe Schmoe White American from having a chance at using it to incite.

It's a hard problem with many a false-positive to be had in ways that will befuddle, outrage, and amuse.

That's an excellent question, the ml modelers should ask those questioners themselves. There is a comment in this thread that in Spanish languages asking for a black ink cartridge (cartucho de tinta negro in Spanish) for a printer is automatically blocked because the word negro in hate comment is the word that has most signal for the classifier

Such nonsense as AI moderation will force everyone to be good but good only in terms of some very narrow cultural norms with no respect to other cultures.

And since practices like these only growing it will become norm in near future.

Not only AI.

Spanish speakers have gotten into trouble for using the Spanish word for black in other contexts, like:

British audience misunderstanding Spanish phrase by an Uruguayan player.

https://www.dailymail.co.uk/sport/football/article-9104039/M...

https://elpais.com/deportes/2020-12-23/gracias-negrito-la-ac...

German players misunderstanding Spanish word:

https://apnews.com/article/sports-2020-tokyo-olympics-africa...?

https://twitter.com/Romein0omatic1/status/141646122833191322...

Yes, it's just sad that some people start judging beforehand, implying malicious behavior rather than misunderstanding.

And other people want to be offended, even when there is no insult, just different cultural norms.

That last link is the distilled essence of Twitter to me. A white woman from America telling a black person that he doesn't understand what a black person's perspective would be on the topic of racial slurs and how they [mis]translate across languages.

Reply on news.ycombinator.com