Anatomy of an online misinformation network
journals.plos.org
journals.plos.org
The paper also finds that many misinformation accounts are not bots. Currently, misinformation is not a bannable offense for most social networks. Twitter, Facebook, and YouTube leaders have said, verbatim, that they don't want to be "arbiters of truth". Of course, that leaves open the question of what responsibility they have and what actions they should take.
Edit: let's have a look at https://twitter.com/afneil/status/1001158920663109632 - Andrew Neil is Chairman of the Spectator (important "serious" rightwing politics magazine) and a BBC presenter. He's accusing the NYT of "Stalinism". One of these has to be wrong.
Oh, at least ten thousand of those retweets were post-correction, as is usual on Twitter.
There might be better solutions. One idea is to pay nonpartisan organizations to fact check viral posts. Another is to reduce the use of engagement as a metric, which tends to promote emotionally charged rather than factual content. For example, in January Facebook committed to prioritizing "meaningful social interactions".
There are lots of people who will "question" endlessly the meaning of "misinformation". Trying to make exceptions for that .01% of the time a "gray area" can exist, while ignoring the real harm to society caused by the 99.99% "deliberate misinformation".
I guarantee you, the responses to your comment will be exactly that.
I would compare it to science-denial. Like people who refuse vaccines because of the tiny risks involved, while ignoring the outsized benefits.
People who continuously debate the nuance of "what is" "misinformation", while ignoring the harm it does, is no better.
And while one singular bit of misinformation may seem harmless, goofy, and opinionated, ie. Obama is a muslim, crowd size, "Uranium One" etc... A constantly stream of them practically creates an alternate reality for people who believes them, and that is extremely dangerous.
Evidence can change and something regarded as false yesterday, can be true tomorrow. Congratulations, you committed censorship.
Take your example of vaccinations. Huge topic, even in my country which is at a 96% protected rate. Misinformation would evidentely be that anti-vaxxers are a significant problem that should be talked about and that we seriously need legislation to enforce vaccination.
Blatently false, the numbers show it. Having any other opinion is pure science denial.
That said, it can be a problem in other parts of the world. So what should be the reference here?
You aren't even able to develop a theory about the damage of misinformation that would withstand scientific scrutiny. Maybe only because of lacking data than there not being a correlation, but still.
So in the end I would conclude you only want to censor things you do not like.
If you are more certain about your point of view and are able to articulate it, maybe more people will believe you.
> Blatently false, the numbers show it. Having any other opinion is pure science denial.
For measles, the herd immunity threshold is 93-95% [1].
1. http://www.who.int/immunization/sage/meetings/2017/october/2...
Even with legislation, you wouldn't achieve a 100% protection rate.
You can also verify the insignificance of this discussion if you look at the victims of measels.
You can always argue that one is one too many and I would agree. But I disagree about the means in case of doubt. But all that is off topic.
Earlier today I saw a thread of tweets criticizing a media figure for promoting the claim that Chelsea Clinton is married to a nephew of George Soros.
This is a relatively easy claim to check; the identity of Chelsea Clinton's husband is not a secret, and Soros' family tree is also known. Her husband is not, in fact, a nephew of George Soros.
Do you believe it is correct, then, to identify a tweet claiming "Chelsea Clinton is married to George Soros' nephew" as "misinformation"? Or would you, in a sense of charitable benevolence to the shifting nature of truth, want to hold off, just in case tomorrow she wakes up, decides she's unhappy, and files for divorce so that she can actually go marry a Soros nephew?
Let's try it more neutral. Water is an aggregate state of carbon dioxide. Is this misinformation? Yes. Do I want social networks to be chemical teachers? No.
The .01% is enough of a basis for a lawsuit. And grandstanding by the right claiming the company is biased (never mind that almost all "misinformation" is right leaning, so it's likely the .01% false positives are also likely right leaning). I would absolutely not want to be taking on such risks from a business perspective. If it's societal good we're aiming for, let society (elected officials) figure out how best to achieve that. Do you really trust for-profit organizations to not have their "editors" swayed by customers?
Which by their very nature subvert and stretch the simple true/false dichotomy.
Unfortunately, even if those pushing deliberate misinformation are banned, then the problem will keep going since others will spread the information because they mistakenly believe its true.
If you then decide to ban all misinformation (because false information is a problem even if it is a mistake), that goes back to the difficulty of deciding what is misinformation - people can cherry-pick true information in a way that gives a misleading impression, or if someone makes a statement that might be true, but there is no evidence for it, do you have to assume that it is false?
I agree that there is a problem here, but I don't know that banning misinformation is the right way to solve it - there are too many gray areas, and it grants _a lot_ of power to social media companies to let them determine what is true.
The interesting part is the mutual reinforcement of targeted content and what we may call "idiosyncratic truth," since both refer to distinctive group boundaries and the respective framework of shared world view. Social media will always reinforce such bias, it's part of the setup.
Lastly, if we acknowledge that there's no way to identify an unbiased (positive) truth, it may be also hard to identify bias at all. It may be possible to identify misinformation which is spread arbitrarily as part of a broader strategy (then better called disinformation) but even then it may be hard to reach consensus. (Elefant in the room may be the UN presentation proceeding the last Golf War.)
Which really is the argument that falsehoods are [some portion of the time] rooted in wrong assumptions and premises.
I would argue that this only strengthens the statement that facts exist.
That some people (here on Earth) want to claim that 2+2=5 because they have invented their own number 2 (which they conveniently withhold), is precisely the kind of misinformation being discussed in the thread.
(Also: Let "x" = 1", "2" = "++x" and "4" = "2(++x)". Results may depend on your C implementation.)
Aristotle defined truth essentially correctly: "To say of what is that it is not, or of what is not that it is, is false, while to say of what is that it is, and of what is not that it is not, is true."
Are you saying that reality is also ideological (quite a radical claim) or simply that perceptions of reality (and therefore what people think is true) are ideological (which is a rather tamer claim)?
I think that is a great example of something which is obviously untrue.
Deciding who we should and shouldn't silence is not the solution. Whatever one chooses, some propagandists are going to win. Giving people the tools to determine the veracity of a statement themselves is (usually this is simple common sense, the ability to reason, some knowledge of rhetoric and an understanding that personal bias in prevalent in themselves and others - that it's human nature). Throw in a bit of knowledge about politics (both domestic and international) and armed with these tools I'm confident the majority would be able to come to accurate conclusions on their own.
This will never work in practice, truth is not a popularity contest.
I'm glad someone is making a public/academic attempt to measure this, because it's hard to imagine just how easy it is to spread disinformation.
What would you have done differently?
But the qualifier "bad" is ensconced in lofty vocabulary. The reality is this is likely a generalizable analysis of how to attack information networks regardless of alignment. All of the high-level adjectives ascribed to these networks are window dressing, related to a separate discrete task: target selection.
After that, it's just take down operations.
The inverse activity is conceptualizing "take-down resistant" information networks, at which point, it becomes evident on closer inspection that all of this is just the throes of an arms race. So let's just cut to the chase, and stop thinking in terms of cat and mouse games.
This is all simply reduced to harasser/harassed relationships. Calculate motives, look at what the objectives of the belligerents are, and instead of feeding into the meat grinder, aim for greener pastures. Go where everyone else isn't.
Given that the content may be arbitrary, looking at the content alone is not enough. Analysis of at-risk "swing voting" audiences would be the qualifier for an offending orchestrated botnet.
The intent of these botnets for the past victory was to "milgram" people into voting Trump (influence opinion, by surrounding and immersing of easily swindled individuals in currated content, many sources creating a magnitude perception for illusory peer pressure) [0] so, the content is nothing if it isn't seen.
Whatever the case, this is last year's tactic. It's possible that it may never be used again, in anticipation of adaptaion to defend against it. That's if you buy into the hypothesis that this is why people voted the way they did in the 2016 election, and that external technical influence was truly responsible, and even capable of such a manner of attack.
To be honest, it also sounds like a kissing cousing to the hand-wavey premise of flashing inserts of suggestive hypnotic single-frame advertisments (BUY MORE POPCORN fnord) in movie theaters to provoke subliminal brainwashing so that people buy more popcorn at movie theaters.
The solution to misinformation in a community would likely be community management/moderation, but that can't scale to the size of something like Facebook or Twitter. Hence the awkard tech 'solutions' that don't work.
Why is twitter not fixing these issues?
Maybe religious people aren't the one-dimensional voters that you imagined them to be. But if you were to admit that, how would you slam both Trump and non-atheists in the same post?
Your point would be valid if religion was a low priority hobby.
It isn't, so (a subset of) [religious voting block] are just religious when it suits them, ie hypocrites.
If you could get over your contempt for people who see things differently than you do and actually talk with some of them you might find that a lot of people were voting against Hillary instead of for Trump.
> ''The problem is,'' he said, ''there are a few hundred women every year who have personally agonizing situations where their children are born or are about to be born with terrible deformities, which will cause them to die either just before, during or just after childbirth. And these women, among other things, cannot preserve the ability to have further children unless the enormity -- the enormous size of the baby's head is reduced before being extracted from their bodies.
> ''You know, Hillary and I, we only had one child. And I just cannot look at a woman who's in a situation where the baby she is bearing, against all her wishes and prayers, is going to die anyway, and tell her that I'm signing a law which will prevent her from ever having another child.''
Clearly a monster of pure evil. And his wife too.
Whereas her opponent supported torture, dehumanized immigrants every chance he got, has 5 children with 3 different women, bragged about cheating and sexual assault, lied as easy as breathing. Admittedly after the election, but supported a pedophile, although he joked about dating his daughter years before.
I fully stand by exactly what I originally posted. As a raised pentecostal Christian, if that should matter at all.
Their stock up 100% from the 52 week low. Nothing's broken.
How could disinformation running rampant on your platform be bad, if your stock is up 100%?
Shareholders are happy, the market's happy, who cares if your pipe is carrying a cultural toxin, as long as that pipe is also delivering revenue and shareholder value?
(Of course, I disagree with what I wrote above. But it lays bare why this hasn't been addressed. Also applies to other social networks, owners of digital, print, and broadcast media outlets.)
Either they'll fix it, or at some point, consumers will jump ship to a competing platform that includes appropriate protections against these issues.
I currently use few accounts, which are spreading false-news, as indicators with great efficiency. A bit of automation and a browser plugin from trusted source (like Bellingcat) can help to fight that problem.
Fake news isn't there to drive user engagement, it is there to derail societies and institutions that underpin those societies.
I could be wrong, but I don’t think the poster above yours were implying the aim was to take down all of society — I think they were implying the goal of some of these networks is to bring down particular targeted societies.
I tend to think you’re both correct. I think it’s a combination, some people are into driving traffic through clickbait and some networks are amplifying divisive subjects in an attempt to make The Others appear to be unreasonable.
There are certainly many other networks taking part as well. There are definitely other networks of unorganized regular citizens who are also intentionally pushing misinformation in order to make their ideological rivals appear to be completely irrational. Whether these citizen groups are a consequence of the former groups I’m not sure, but knowingly pushing misinformation is definitely not isolated to governments and clickbait sites. We seem to be overrun with unapologetic and ideology driven liars.
This is somewhat of a tangent but still related, we can see an active attack on science and higher education right now and it’s pretty clear why, many of these ideological driven people instinctively know they’ll never return to their former glory when science and an educated populace stands in their way. I think we’re going to discover many of the root causes of the misinformation campaigns stem from the same battleground - many groups and individuals who instinctively know science and the-closest-thing-to-truth-we-have stands in their way.
Your last paragraph about science could be about all sorts of groups but it sounds like you have your personal political enemy in mind. You could be describing Muslims who are opposed to science's revelation that women are as smart as men. Or it could be about liberals who are opposed to science's revelation that the smartest men are smarter than the smartest women (male IQ has a higher standard deviation than female). Or is about feminists who are trying to suppress science that shows that women aren't as hard-working as men? Or is it about transgender activists who are trying to suppress science showing that women are not men? I wouldn't want to censor any of these people. Who am I to say what the one true truth really is for all time? Even science doesn't know that.
What I really think the GP meant, but didn't say because when you actually write it down, it sounds ridiculous, was a conspiracy theory like "Putin wants to destroy America so he supported fake news to get Trump elected and since Trump is obviously bad, he'll destroy America.". As far as I can tell, that's what the opposition to fake news is all about - Trump winning the election. That's certainly when it started anyway.
At the moment people can publicly and on-the-record spout the most outrageous bullshit and even when they get proven wrong there is no repercussion. none.
On a personal level, just cut those people out of your life. Refusing to interact with people who knowingly lie and knowingly spread misinformation is the only rational choice. Time is our most valuable resource, spend your time on projects and people who will carry us forward rather than wasting time on people who are just worthless timesinks.
Is the claim that there are only the right-wing sources for these misinformation networks, or is this just one network they examined?
Yesterday, twitter was full of pictures of the immigrant kids that Trump put in cages. This morning, we find out that the pictures were taken in 2014. It's a difficult problem because the retractions are never broadcast/retweeted as prevalently as the original fake news.
This is a difficult area, and the stakes are so high that the arms race will doubtless continue. Because the stakes are so high, you are competing with organizations from allied countries as well as non-allied countries, and from multinational corporations and industry groups, all throwing tens of millions of dollars each on think-tanks and lobbying groups, whose main activities are to publish source material that can be used as supporting "facts" for various news stories.
Perfection shouldn't be the objective. If it's better than what we have then it's worthwhile.
> do tend to lean left
Citation needed.
And if you know of a fact-checking resource that has less bias, please share.
> Citation needed.
I think that this is probably a dangerous game to get into; calling for citations is, I think, only likely to be helpful when there is a mutually recognised authority to cite (and, besides, when the claim to be resolved is a factual one—'biased' is not a characterisation that can be proven or disproven without some really careful definitions, on which two disputing parties are likely to disagree). Here's the first hit in a search for "Snopes biased". I've intentionally mangled the first component of the URL to avoid linking to whatever kind of site this is.
http://d@ilyc@ller.com/2016/06/17/fact-checking-snopes-websi...
I don't know Daily Caller, but the title alone makes me fairly confident that it's a biased source; but there it is, a citation to back up the claim that Snopes is biased.
In my experience they tend to be biased against truth as it were (and so would happily choose Snopes as a target)— but that’s just the humble opinion of one observer.
The same is true when it comes to source versus fact. A broken watch is still right twice a day
> biased' is not a characterisation that can be proven or disproven without some really careful definitions, on which two disputing parties are likely to disagree)
That happens to be the modus operendi of the propagandists, the liars, to say there is no truth, there's no way to determine it, there's no source more accurate than another. They say 'it's all propaganda', trying to put themselves on equal footing with honest people. It's liars who say, 'everyone lies'.
In fact, humanity has developed many ways of distinguishing truth from lie, fact from superstition, critical thought from ideology, physics from alchemy. That is, in a significant way, the project of the Enlightenment, which led to science, the rule of law, liberty, democracy, and all the things that have been born of them (including the Internet). Not coincidentally, I think, the propagandists are generally enemies of democracy and seek to suppress its effectiveness and the power of the public, in part by undermining its foundation of informed citizenry and open debate by interfering with it and saying it's impossible.
Yes, this was my point. If you (not you personally) are arguing against a propagandist who says that all your sources are biased, then I don't know what is an effective rhetorical strategy—it seems that no-one does, if there even is one—but I'm almost certain that asking for a citation for the claim of bias isn't it, since all it does is shift the question from "who is biased?" to "who can be trusted to judge bias?".
I see your point, but I think the propagandist is only effective if we buy their premise, that there is no way to distinguish between biased and unbiased sources (and information). My point is that there are many ways to respond, that it's not at all a new problem, and that the solution is reason, in the Enlightenment sense. The propagandist says it's all arbitrary faith, but reason always has been their downfall (which is why propaganda often appeals to hatred, to crowd out the application of reason).
(On one had, this comment and my GP comment are written so loosely and broadly that it's a bit embarrassing; on the other, I think my point gets across and I don't have time to write a treatise.)
It is useful, in this regard, to read about the Institute for Propaganda Analysis, and to read Aaron Delwiche's http://propagandacritic.com/ .
The above "Citation needed." is an opportunity to point out the concrete reasons (if any) why bias is supposed. If the only evidence is from sources that are widely questioned then that itself is something to note.
And as far as bias, a quora post [2] seems to give the impression that Snopes was heavily favoring Hillary Clinton through the campaing.
So I'd like to turn this around: could those that believe Snopes is not biased give some kind of citation?
[1] https://www.forbes.com/sites/kalevleetaru/2016/12/22/the-dai...
[2] https://www.quora.com/Is-Snopes-biased-Why-do-some-people-be...
Disinformation is built on shared assumptions with weak evidence.
When "everybody knows" the media is biased, all the effort by reporters, fact checkers and editors is instantly discredited and disinformation gets equal standing.
"Citation needed" is a challenge to examine the evidence. Sure, you can find citations to disinformation that "substantiates" disinformation.
It's not about the citation, it's about examining the credibility of the evidence.
No, it's not perfect and it never will be. But letting "everybody knows" statements go unchallenged is how we get to a world where "nobody knows" and evidence doesn't matter.
So you want a link snope or politifact link? Even if people provided citations, wouldn't you reject them because it isn't snopes or politifact ( the approved "fact" finders )?
The fact that much of the liberal media and institutions have rallied around snopes should indicate that snopes has a left bias. The fact that you ( likely a left leaning individual working in the media ) support it probably indicates it as well.
> And if you know of a fact-checking resource that has less bias, please share.
Everyone is biased. Especially when politics, money and propaganda is involved.
The issue with news media isn't facts. It's the spin on the facts.
Look at Trump backing out of the NK meeting last week - which is a fact. But look at how CNN and Foxnews spun that.
CNN : "Trump is backing out of NK meeting because he is a bumbling fool failing at NK negotiations"
Foxnews : "Trump is backing out of NK meeting because he a deal making genius who won't let NK screw over the US again."
Same "fact", but wildly different spin. Fact checking is never going to solve this issue because "facts" are only a small part of the news industry. The core of the news industry is to push propaganda and to influence the people to think a certain way.
Also, propaganda isn't necessarily a bad thing. Every country needs propaganda to exist.
If you are interested, here is a former TIME magazine editor speaking on the topic of propaganda, history and news.
Citation needed.
To Trump supporters, The New York Times is an institution that spreads lies with abandon, and they get to operate enforcement agencies at the moment. The belief that it's seeking the truth in good faith is essentially just liberalism. Same for Fox and friends.
Given how incredibly entrenched some power structures based on lies are it looks as if my plan is mostly failing. Education budgets are being cut left right and center and the quality of education has been steadily trending down since ~1950 or so.
Some are so sure of their world view that they just assume their conventional wisdom is correct about specific subjects that they clearly (to someone who has) have not studied. I know I was back in 04 when a friend mentioned something to me that I thought was absurd, but is now common knowledge for anyone understands F=ma and has checked that specific fact (that I'm deliberately not mentioning). For example, look up the art collection Skippy's brother keeps, or that shop owners (now hidden but thoroughly archived) Instagram account.
"Fact checking" sites are about as unscientific as it gets. They have two obvious flaws, first they generally only "check" facts they can "debunk", second, they often frame what they are checking in a way that lets them use a corner case.
The right way would be to just list data, things others can independently check and allow the reader to draw their own conclusions.
This paragraph is almost incomprehensible to me. Is it meant to be read with a lot of background knowledge, or am I just being dumb (or both)?
https://www.huffingtonpost.com/2008/11/25/john-podesta-talks...
Also that statement applies to you as much as it does anyone else.
Not sure why he is too scared to just say this. I guess in the cold light of day it sounds pretty stupid. Perhaps if you wrap it up in code words it sounds a bit better, or at least makes him feel special.
The Russians really did a number on the US didn't they.
Why is that? Is your conviction in what you believe that weak?
So, one last time, share your views and maybe a link or two. No deflections, no comments on your irrelevant theories about peoples worldviews (obviously born by reactions to when you behave as you have in this thread) and no childish code words.
Thanks.
Aside from documenting to what 'Skippy' refers, you don't seem actually to have suggested anything in particular—at least, not clearly enough that I can tell what it is. (It's quite possible that I would be defensive if I understood what you were suggesting, but I can't now because I have no idea against what I'm trying to defend.)