The effect of deplatforming hate organizations on their online audience
pnas.org
pnas.org
It's easy to ban a bunch of people and come up with an estimate of a large amount of hate reduced. "These hate-mongers posted X times per day to an audience of Y people, therefore we stopped Z hatred-incidents per month. Big progress in stopping hate.". That's the stupid man's interpretation of the available data.
It's a lot harder to measure things like the feeling that people who already feel disenfranchised about life feel when they can't even vent about stuff on the Internet and what that leads to. The people with the ban-hammers are doing nothing but creating further and further resentment and confirming whatever biases and conspiracy theories that many people already feel.
You had just got done critiquing the article in a clear and cogent way, then you go off and assert something for which you have no evidence.
The existence of these online groups, they get people, they turn good people into bad people. We have friends and family we have seen it happen to. So perhaps they are not ONLY causing resentment among those already converted, but perhaps also preventing further conversion.
You don't necessarily need, nor does one necessarily exist, a scientific study or formal citation for something fairly subjective/opinionated that you can give a justifiable explanation for.
If somebody is blaming some group, say the French, for controlling everything from the media, taxis, the money supply, politics, pharmaceuticals, real-estate, Hollywood, the Fast Food Industry, and Silicon Valley and suddenly get removed from all of polite society, it doesn't take a peer-reviewed scientific study to suspect that they're likely going to be having even darker thoughts about the French and feeling that this confirms their theories that the Frenchies are out to get them.
> The existence of these online groups, they get people, they turn good people into bad people. We have friends and family we have seen it happen to. So perhaps they are not ONLY causing resentment among those already converted, but perhaps also preventing further conversion.
1) You just got done critiquing me for trying to assert something with no evidence, but then did quite a bit of that here. Should I demand a citation and scientific study for every opinion here?
2) You use the phrase "bad people", and maybe many of them are, but there's a wide range of views out there that are labelled hate that might not be so.
There's also a wide range of views out there that are correctly labelled hate.
Wow there. I think the who point of hate is you're taking it out on people? Hatred leads to violence and interestingly, much like violence you shouldn't use it as way to vent your frustrations. Much like how drunken husbands shouldn't go home and take out their frustrations by beating their family.
> creating further and further resentment and confirming whatever biases and conspiracy theories that many people already feel
I say that's a good thing, let the hate fester and consume the hateful without infecting others. It's like a disease that spreads from host to host. If you choose to be hateful the enjoy your misery or be violent and enjoy the consequences.
Name me one issue that requires you to hate people in order to resolve it.
Anger has been beneficial. Workers had anger to get us the 40 hour work week. Women had anger to get them the right to vote. Black people had anger to get the right to vote.
I agree there is a problem with the hateful festering and spreading. I think the solution is societal safety nets: universal basic income, universal healthcare, free college or trade school.
Hate in this context also means hatred against others based on factors about themselves they cannot change and based on things they didn't cause, because they are a member of some group
But if that random person is removed from Twitter and starts seeking out like-minded people, they're going to form some underground community with no pushback against the worst ideas.
2) Since you seem to hate Nazis so much, why are you adopting the digital equivalent of their tactics? The Nazis famously burned lots of very Jewish books, right? Society used to say that this was one of the examples of why they weren't so great, and I grew up hating the idea of burning books because of this. I can't understand how using their playbook suddenly became doubleplusgood.
That's a real population of extremists. Having them hang out on HN does not make their numbers dwindle.
And no, I do not think that banning Nazis from forums is "the same playbook" used by the Nazi Party.
I'm always open to at least talking to anybody, whether or not they're ISIS members, Nazis, Communists, or (may Allah forgive me for even uttering this word) Canadian.
Talking is the road to peace. When words stop, there's only one thing that can possibly come next.
You are free to go to their spaces and talk to them if you want. But there is no moral need or practical good to permit their hatred in all spaces.
What if the more important metric for a good society was something else?
What if removing "the bad guys" from society makes them feel like they're not part of society and have nothing else left to lose so it actually pushes them to violence?
I'm not saying I have all of the answers here, but I do strongly think that your general approach here is ignoring the human side and focusing on the wrong metric for progress.
Starting with H1, it's quite clear these are not their definitions, but they're your misinterpretations of their definitions. Is there a reason you're misrepresenting the conclusions of the study, and doing so as uncharitably as possible? Is it possible it's "the stupid man's interpretation" because it's not at all their interpretation of their results?
> It's a lot harder to measure things like the feeling that people who already feel disenfranchised about life feel when they can't even vent about stuff on the Internet and what that leads to. The people with the ban-hammers are doing nothing but creating further and further resentment and confirming whatever biases and conspiracy theories that many people already feel.
It seems weird to note that it's hard to measure while simultaneously expressing complete confidence in the data-less conclusion you seem to want to be true.
nobody is preventing people from healthy venting... that's some kind of ridiculous strawman.
Well, then why don't they vent about this stuff in person? Surely that's an even better marketplace of ideas.
Which is why so many people are upset at the level of censorship happening online.
Reddit permanently banned my account for _upvoting_ covid vaccine hesitancy (which is turning out to be accurate).
Anyone who doesn't like this submission being flagged can always start an offsite dedicated to advocating for even more extreme censorship.
And yet, in every argument defending hate speech under the principle of absolute free speech, or denouncing "the left" or "the wokes" or whatever the bugbear of the moment is, they seem to know exactly what hate means and how to apply it.
Anyway, since no one is going to bother reading TFA and this thread is probably going to be thoroughly derailed and flagged, the effect seems to be that deplatforming works:
We study the effects of six network disruptions of designated and banned hate-based organizations on Facebook, in which known members of the organizations were removed from the platform, by examining the online engagements of the audience of the organization. We show that on average, the network disruptions reduced the consumption and production of hateful content, along with engagement within the network among audience members. The results suggest that strategies of targeted removals, such as leadership removal and network degradation efforts, can reduce the ability of hate organizations to successfully operate online.And, furthermore, the finding is akin to a form of survivorship bias. 'We found they stopped talking to each other and promoting hate online.' Okay, and what did they do next? Did they just stop hating people? Did they figure out a way to start meeting in person without Facebook to mediate the conversation? Did some of them become increasingly isolated that they decided to act out? Did some people who weren't members of the hate group happen to see this form of censorship and start to change their own minds? I'm not saying any of these outcomes are what happened or are inevitable, I'm wondering how they can call this real science when they don't even bother looking at the other effects outside of their water-is-wet conclusion.
Given that the target audience they were studying were users of the platforms, and the effect was the consumption and spread of hate speech within that population and on those platforms, I don't think that including n+ order effects across every other facet of society would have been necessary, or even possible.
I find it odd that it's perfectly legal to be a group with illegal intent. That's the only loophole that needs to be closed.
If a group is deemed to have "intent to harm and endanger others" then anyone who affiliates themselves with them should be deemed likewise, and attended to.
So the issue is of pre-crime. A crime has to happen and speech isn't a crime (although censorship should be, we censor them instead).
But if members of a group have already committed crimes and crimes of hate, then that should be enough to legally impose restrictions on the group and its members, including those that haven't. Maybe not arrest them immediately, but monitoring etc should be perfectly legal (and maybe done already).
All any member has to do is unaffiliate themselves to free themselves from being high risk and attended to by law enforcers.
> then that should be enough to legally impose restrictions on the group and its members
This is called "keeping a public nuisance" and it's illegal in many jurisdictions. It's not limited to hate crimes.
These conversations about online hate groups would be more productive if people knew what the law was to begin with.
Also I am not claiming I know any laws.
I am claiming the law needs to be adjusted, not censorship standards.
What's that, precisely?
What is your goal with this line of argument. This looks a lot like devils advocacy. Like, what good comes from someone saying: "There's actually no such thing as hate-organizations." (Something you never said).
According to some people, they’re advocating for actively harming people.
According to others, they’re advocating for preventing harm.
It depends on how they do it. Advocating for something is not hate
Here's a question for you: do you consider so-called "TERFs" to be a hate-group?
And here's another: what do you do with the fact that the very notion of violence is being redefined to include such things as misgendering people?
So I ask you this: are you really suggesting it's unreasonable to view this kind of research with caution and circumspection?
Because if it's a logical OR, butchers might fall into the first 2 categories.
And the last part is not very enlightening to define a hate group as it also uses the word hate again
(Posted in the expectation that someone will be along soon with a bucket and mop).
What is the purpose of this line of argumentation? Would you take the position that there are no such things as hate-organizations or hate-speech?
The world is plainly better off with fewer organizations like the KKK and ISIS. Do you agree? That seems like as milquetoast as one can get.
Are there organizations that you feel are labeled as hate-groups but are not? What hairs are we trying to split?
Also, I don't this paper is talking about the KKK or ISIS since I don't think they have a had presence on a social platform for quite a while.
Obviously also there are plenty of organizations that are labeled as hate by some people and other people disagree with it. Look up this week's controversy about SPLC for some examples, but they are in no way unique. You think of it as splitting hairs because you can't imagine how people can disagree with what you imagine to be obvious. But it's not as obvious as you think.
If a person or organization calls for violence or the destruction of property, they are a terrorist organization. If the reasons they give for this are based on identity they are additionally a hate-group.
Seems pretty clear to me. A world with less moral clarity here is one with more (necessarily?) violence and destruction. Why do you think this line of argumentation is either morally permissable, or even virtuous? What argument convinced you?
Are Martin Luther King or Malcolm X terrorists?
US Army has left the chat. All the militaries in the world joined them. The police and the FBI are not far behind.
> Why do you think this line of argumentation is either morally permissable,
Which "this" line? Questioning your definitions? I don't think I need anybody's permission for that.
But wait. Let's say the KKK 3.5, freshly reorganized, does not call for violence and destruction of property of Jews and black people. Make no mistake, they still hate them and think they are sub-human and should be oppressed and ultimately eliminated, but they are now vowed to act through strictly non-violent means - education, propaganda, investment, protest, boycotts, electioneering, lawmaking if they could make it - but no freelance violence. And, also, they are called LLL now, just because. By your definitions, they are not a hate group anymore? If yes, then what criteria did you use and why didn't you initially include them?
It's a lazy, imprecise term that is only ever used in political spats, never to address actual intergroup violence. Personally I suspect the demand for hate groups to exist - thereby justifying the many orgs and careers built on countering it - far exceeds the genuine supply.
The Babylon Bee was a satirical news website, like The Onion, that was deplatformed (censored) from Twitter for making a joke about a trans person.
Hate is often defined on these platforms as speech that can cause offense to some people.
It's not relevant to this paper.
There was a joke attached, not that this is at all relevant — they were using humor to highlight the absurd.
They know that "hate" can be redefined in any manner as they wish, and if they can correlate hate <=> censorship is beneficial, and they can correlate anything-I-don't-like <=> hate, then they can correlate anthing-I-don't-like <=> censorship is beneficial. But they want to pretend you're the "sea lion" in some moronic comic as an ironic shorthand to disrupt and short-circuit a serious conversation on censorship.
1: https://theconversation.com/the-inherent-racism-of-anti-vaxx...
2: https://www.rollingstone.com/music/music-features/eric-clapt...
3: https://www.theedgemedia.org/purebloods-anti-semitism-white-...
Group X is a hate organization if it explicity or implicity advocates, promotes, supports, or works toward the destruction of another group Y, with one or more of the following being true:
- the reasons for the destruction are based on criteria that Y was born with and cannot change,
- the reasons for the destruction are based on unproven, unprovable, or false claims, by X, targeting Y, that Y will destroy X. X may claim Y will destroy X through Y's very existence, through specific or vague planned action, or through attempts to rally, proselytize, or persuade. Claims may be but not limited to being scientific, economic or religious in nature.
- group X believes it is receiving a mandate to commit the same to Y, due to an unproven, unprovable, or false claim that it has received a mandate, or one or more risen ringleaders in X have issued such a mandate based on a similar unproven, unprovable, or false claim.
and often:
- Ringleaders of X may be leveraging/exploiting/extorting one, some, or all of the following aspects of group members: emotions, faith, economic status, state of happiness, or threshold of accepting something as true without personally witnessing. The purpose of the manipulation is to align members of X towards action. Propaganda-like techniques may be employed.
I reckon for most; their hate is based on emotional manipulation over an internal drive. This is what separates their "leaders" who are doing it for a specific end result, and their "followers" who are reacting to whatever comes across their feeds. Which is how they leverage this, it's easy to emotionally manipulate those who have no idea what's going on, they don't even recognize the ads/recommendations they're getting have served a specific purpose.
Keep in mind I'm European, and not American. I think we have a different view on this. Especially e.g. countries with a coloured history of hate speech like Germany.
1. Ideas need to be open so they can be challenged: These platforms don't create discussions, they create echo chambers. Nobody of their right mind engages with the crazies online as it's obviously a waste of time. Try to go to Facebook and debate with any of their "Flat earth" groups. It's absolutely futile. These "debates" are better for recruiting new people into hateful ideology than building understanding.
2. "Free speech" something or another: There is no such thing as "free speech" because there are already limits to speech. We've already drawn an arbitrary line on what's legal and harassment, death threats, racism, discrimination, defamation and multitudes of other types of speech are already controlled. If something is illegal to say to someones face, why should it be legal to say online? Not to mention that discussions about free speech spans borders and laws are different across them.
3. Who defines what's hate speech? I'm not an American, but I am at the mercy of your platforms. I for one would much rather that at EU-based body, or my government (I know this sounds crazy for an American, but we have more trust in our institutions) decides what's OK to say rather than tech billionaires like Elon Musk, Mark Zuckerberg, or Sundar Pichai. These people are already making moderation decisions, which is inevitable with large-scale social media. Make these policy decisions openly. Make them democratically.
Large scale social media platforms are cesspools already. Unmoderated social platforms turn into cesspools by default. Moderation of these platforms must be moved out of private companies and into the public sphere so we as a society can agree on what is right, and what is wrong.
They say the people get the government they deserve though.
I don't believe that my government will turn to tyranny suddenly when it's generally had a pretty good history of not going off the rails. I don't buy the slippery slope argument/fallacy.
>They say the people get the government they deserve though.
Touche. I'd be more concerned about this in the US with your highly polarized yo-yo politics.
As far as your government turning to tyranny. I can't think of many governments that turned to tyranny multiple times. They are usually removed, or stay in power. "My government is not currently tyrannical so I'm confident it will stay that way", is a strange way to think if you've read much history, imo.
Yes, and I'm arguing for stricter enforcement of current laws around these topics. I'm not saying that people can't discuss immigration policy. I'm saying that people shouldn't discuss immigration policy through inciting to violence and racism. And I'm saying that Elon Musk is absolutely the last person I'd want to be the arbiter of what speech is legal or not.
Yes, the Norwegian government has some not so great history of discriminating against the native population hundreds of years ago, the Roma people last century, LGB people ~30 years ago and most recently drug users. But thinking that "Because there's always a slim chance my government can go rogue we should minimise its power" is a strange and uniquely American way of thinking by the way. My views are not uncommon in Europe. But they are at odds with the US/Libertarian slant you find here.
>I'm saying that people shouldn't discuss immigration policy through inciting to violence and racism. And I'm saying that Elon Musk is absolutely the last person I'd want to be the arbiter of what speech is legal or not.
Yes and I still think I'd rather be kicked off twitter by Musk than locked in a jail for saying the wrong thing. I honestly have no idea why you believe that the members of your government are more fair/just/whatever than anyone else, including Musk. They're not, and that's why they should have limited power.
I'll just dip out of this conversation right here. We have a fundamentally different view of how society is built up, functions and should work.
> Your views may not be uncommon, but they're certainly not universal, and even if they were, that wouldn't make them correct, or even well thought out.
Touche.
At the risk of seeming like I need the last word and further risk of sounding very cliche, there is a quote by a man named Martin Niemöller, a German man, about a German government, that I believe, tells us in no uncertain terms, that the slippery slope fallacy is not a fallacy AT ALL, and that we should take government powers and limits extremely seriously.
> Germany have criminalized basically any positive display/speech about their past tyrannical government.
Germany went through the worst case scenario of hate speech. Hate speech is strictly regulated there because they know the dangers it poses and how it can make normal people into monsters. While many factors went into the rise of Hitler, hate speech was a fundamental part of how precisely Hitler got to power. Hate speech enables populists power through simple emotional messaging. Any idiot can do it, and many do. And lots of people fall for it. I don’t want emotional messaging controlling our politics (precisely like what’s happening with the culture wars in the US) I want boring, research backed, evidence-based policy made and championed by bureaucrats and domain-experts with decades of experience.
There’s a reason hate speech is being researched and platforms are being forced to deal with the unmoderated hate speech of the 2010s. It has vast political power in the wrong hands. And it inevitably leads to shit policy.
I 100% support discussion about immigration or if identity politics has gone too far. I 100% support the deplatforming of voices that stigmatise, discriminate, silence and incite violence towards these minorities on major platforms.
"First they came for the socialists, and I did not speak out – because I was not a socialist. Then they came for the trade unionists, and I did not speak out – because I was not a trade unionist. Then they came for the Jews, and I did not speak out – because I was not a Jew. Then they came for me – and there was no one left to speak for me.”
Which is weird because the US government is easily in the top 5 rogue governments around the world.
I have an earnest and legitimate question for you and everyone else of this mindset: Do you believe that these social networks, the media, and the government will always be on the right side of history?
I don't really understand being a proponent of arrogating this enormous control of communications if there is a nonzero chance that the bad guys will be defining the terms of service ten or more years down the road.
Do you guys who advocate for this really believe that the risk of, for example, a Christian revival is precisely zero? That control of this apparatus would literally never shift, and the speech of your LGBTQ friends would never be brutally repressed by it?
That's the only thing I can possibly see as justification for being a proponent of building and normalizing the use of a censorship machine: the solid, perfect, unshakable, 100% conviction that the bad guys would never, ever, get their hands on it.
But if that's the mentality, I don't understand, because history does not support this degree of confidence.
Yes, I believe the local conditions where I am (Northern Europe and by extension the major powers in the EU) will stay stable enough for the foreseeable future that this will not turn into a 1984 scenario. I also understand why people of the United States may have a different view on this considering the unstable political situation you guys are in. I also want to make it clear that this instability is another argument on why relying on American tech platforms to moderate speech in the EU isn't a great option either.
> Do you guys who advocate for this really believe that the risk of, for example, a Christian revival is precisely zero? That control of this apparatus would literally never shift, and the speech of your LGBTQ friends would never be brutally repressed by it?
Yes, I do believe that it is exceedingly unlikely in Norway and the EU generally for the foreseeable future. Far less likely than the US at least.
I think a major point that you need to understand to empathize with this perspective is that capitalism and technological advance is viewed with considerably less optimism and trust in the EU.
Facebook, Twitter, Reddit, YouTube, Google (Search). All of these companies have a massive impact on the European public sphere. And none of these companies give a shit about the content they are promoting. How many Norwegian speaking moderators does Facebook have? How many Swiss? How many Lithuanian? How many Romanian? I bet way too few, and I bet their policies are more hands-off than hands-on. The more hands-on the more unpopular a platform becomes. I also bet that most moderation decisions are made by algorithms and not humans, which definitively affects speech.
While these companies make billions unleashing software that stokes racial and ethnic tensions, polarises, legitimately makes people less informed. Algorithms that structurally amplify the simple, and deprioritise the complex. All in the name of engagement, retention and monetization.
The effect these platforms have on Europe is massive, and I have more faith in democratic governments taking some of the reigns on this than the current state of American tech billionaires and board meetings behind closed doors. Americans have essentially accepted that corporations can use the populace as cattle to monetise. The EU has a different view on this.
The current state of affairs is worse than the chances of misuse in the future is essentially the argument for moving power to the government or enforcing more controls on the platforms in this case.
The rise and fall of the Soviet bloc, Nazi Germany, fascist Italy, Franco’s Spain, and even The Troubles in Northern Ireland — just to name a few!
The majority of the past 75 years in Europe have seen serious, violent, and oppressive political turmoil.
I don’t see how you can possibly take the position that Europe is more stable than the US, much less that Europe has a track record of stability sufficient to justify handing authority over speech and free association to the government without any concern for future abuse.