New disclosures to our archive of state-backed information operations | Hacker News Reader

New disclosures to our archive of state-backed information operations

blog.twitter.com

167 pointschippy95 comments

The actual archive is here: https://transparency.twitter.com/en/information-operations.h... and the data added today was about 5,929 accounts from/by? Saudi Arabia and 1.3Tb of media they uploaded

We recently released a dataset for NLP / disinformation researchers around similar content found being posted on YouTube instead of Twitter which contains images as well as text.

https://media.plasticity.ai/youtube-disinformation-report/

People interested in the Twitter dataset might also find these results useful.

How come there are no information on your site about the founders, team, investors, etc.

It’s hard to take you seriously as the arbitrators of transparency when there is no disclosure about your own company.

We're a YC backed company (YC S17). Nothing to hide, there's plenty of information on us online (CrunchBase, blog posts, TechCrunch, etc.)

If there's nothing to hide, why not publish it where people would check first, i. e. on your website?

There are countries where that's required by law. And while that may be on the rather too heavy side of regulation, I have on occasion rage-quit a vendor's site because I couldn't find this most basic information.

We're based in the United States, there are no such regulations. Many companies don't post information about their founders and investors on their homepage (Stripe, Airbnb, Google, Apple, etc.). You should be able to find information about us just as easily as any other YC company / SV company. We haven't taken any additional steps to hide anything. I'm not sure I understand your concern about this.

It's a pity that they don't tell users that they have interacted with a spam account. Would it be helpful for you to know that you inadvertently shared propaganda?

The cynic in me is concluding that twitter have measured that this would actually cause users to leave their platform.

Again I have no data to back this up, so this is purely anecdotal - the amount of people I see sending re-tweets of probably-a-bot bullshit has me convinced that a huge part of twitter's "success" is built on it.

Hmm, it seems that you are good at hiding your preferences, otherwise propaganda would be much better tailored.

Quite. I think it's also indicative that the 1st goal of this tailored propaganda is not to find the recipients that will be convinced. No, that comes second to _keeping it away from those who will/can question it, and by extension, refute it_.

Nigerian prince does not want to waste his time, he is only interested in easy-to-manipulate people.

I can’t remember where I read this but someone put this forth as a motivation behind the “obviously” fake nature of SPAM: an early filter for people who would call it out. Seems like a plausible theory.

Good spam follows best-practices for email deliverability.

It's not just that. It's about isolating the gullible from people who could protect them by posting refutations.

Twitter in ~2007-2008 was an amazing place when you interacted with actual people.

But now the "frog has boiled" and the humanity has long left Twitter. Practically every interaction between entities on Twitter now is generated by some kind of bot or script that you're essentially trying to communicate with nothing. If there weren't Twitter API limits it would consume more energy and data storage than all the crypto coins combined (slightly hyperbolic). I've left Twitter and I have no desire to return.

However, if Twitter decided to "embrace humanity" and use the API limit system (which already exists) for accounts to increase the value of an interaction they might have a chance to bring back that earlier magic. Scarcity would increase the value, and if you could only post 5-10 new tweets per day you would certainly spend more time considering what you do on Twitter and make it easier for Twitter to identify the sources of noise in the signal.

That would ruin both the "live commentary on TV" and "giant thread" twitter genres. If it includes replies as well .. well, you might as well remove the reply feature entirely and save everyone a lot of pain.

I find it OK, but I'm fairly selective about who I read and liberal with the block button. 500 follows on the regular account, 100 on the Eurovision one (more used for reading hashtags)

Tumblr disclosed which accounts which were controlled by the Internet Research Agency. It was a lot of fun, get to accuse everyone of being a shill since so many people had reblogged from them.

They had gotten tons of reblogs, the most popular one largely posted about black social justice issues and Pokémon memes. Scrolling through isn’t that striking, but the blog did reblog others saying the shooter who killed Dallas police officers could be a mind controlled Manchurian candidate controlled by rfid chips? And some pro Bernie stuff.

This just changes the winning strategy to “have bots post in favor of the position you oppose, then out your human opponents as shills or having been duped by shills”.

Ooooorrrrr you could just cut out the middleman and accuse your opponents of being shills and dupes without any "proof".

Exactly. Which is largely what makes twitter’s attempt to police/publicize this stuff absurd.

Is it even legal for Twitter to disclose activities of US-based intelligence agencies? (I assume that most worldwide agencies play on Twitter in one way or another)

I'm not a lawyer, but I think broadly speaking US law would not prohibit that disclosure. The major exception would be investigations covered under some sort of a gag order. Overall though, the constitutional protections for freedom of the press and speech are strong and well tested in the courts.

Considering some of the confidential things that have accidentally leaked into the public sphere in the last few years, I think it's fairly safe to assume at this point that US/Western government(s) and tech firms have a fairly collaborative relationship.

Some people even suggest [0] this relationship extends into strategic, multi-domain operations, but since such ideas are obviously conspiracy theories (and therefore conclusively false, as per the epistemically infallible Occam's Razor technique), they should be considered only for their entertainment value.

https://en.wikipedia.org/wiki/Information_Operations_(United...

https://www.youtube.com/watch?v=qOTYgcdNrXE

Occams Razor doesnt have to be right.

I dont mean to talk about your speculations, but its not bulletproof method of reasoning

Wait, surely you're not suggesting Occam's Razor is merely an abductive heuristic or something like that, are you?

Sometimes I wonder how much more effective/impactful HN could be in making the world a better place if the hyper-competent people here could maintain a sense of humour while expressing their beliefs, or even consider the possibility that there might be some utility in that.

Twitter information operation disclosures are for US audiences to appease US congress.

>Earlier this year, we committed to the United States Congress and the public to provide regular updates and information regarding our investigation into foreign interference in political conversations on Twitter.

https://blog.twitter.com/en_us/topics/company/2018/enabling-...

Go through the list of countries / actors Twitter has previously disclosures on, Russia, China, Venezuela, Iran, Spain (Catalan separatist), I think Bangladesh at some point, and these disclosures are functionally US propaganda.

I don't think Twitter is an intentional state actor, but who they report on is constrained by limited resources which means they're incentivized to report on (and censor) US geopolitical rivals even though every serious intelligence agency will run information campaigns.

>Information operations on Twitter: principles, process, and disclosure https://blog.twitter.com/en_us/topics/company/2019/informati...

Lawful or not, I don't think Twitter has the balls to expose information campaigns contrary to US interest, which is going to be a recipe for more internet balkanization.

Today's data concerns Saudi Arabia, somewhat contradicting your theory.

Somewhat, though I think Saudi and probably Israel is the perennial controversial ally valley kids have no problem shitting on. Which leads me to question who decides which targets to focus on or which experts help them verify attribution. Part of the disclosure process also includes:

>We also proactively notify law enforcement, our peers, and other relevant state agencies.

Does that include foreign countries? Is twitter coordinating with Iran right now to help fight Saudi info campaign? The entire process chain looks US based or at least distorted by US interests. We know US has Psy-Ops, past foreign influence campaigns (Ernest Voice), and Smith–Mundt Modernization Act 2012 legalized domestic dissemination of propaganda. Either US/Allies does not perform influence operations, or Twitter is not looking hard enough, doesn't want to look or can't disclose. People can judge which options is more likely for themselves.

Why is it so easy to create fake accounts on services like Twitter? If they really wanted to provide a service to humans, what is with making it so that a single group can create 88k fake accounts on a whim.

The alternative would be the chinese model, where you need to type in your ID number in order to be able to play.

Which nobody (rightfully so) wants.

Albeit, it seems like there are loopholes: https://www.abacusnews.com/big-guns/gamers-say-they-can-chea...

The alternative could be a Stripe-like model where as you start using your account more seriously, you need to start supplying more serious identification. Gradual identification or something like that.

What about people in countries were freedom of expression as a very dangerous concept? You would risk endangering them by forcing them to use identification.

This. There are both honest and dishonest reasons to seek anonymity. That tension makes this a very difficult problem to solve satisfactorily, in the context of liberal democracy.

The decision is much easier in authoritarian states, because authoritarians typically hate and fear honest domestic political dissidents even more than they do foreign information agents.

There is a difference between honest anonymity and dishonest anonymity that can probably be exploited to help.

People who need anonymity for purposes such as expressing opinions that could get them in trouble with their government, religious community, abusive family, employer, school, etc., generally do not need complete anonymity. They just need to be anonymous from the adversarial group that would penalize them for their comments.

Such people will usually be able to have contacts that they can trust with proof that they are legitimate, and who are sufficiently outside the influence of the adversarial group that they can openly vouch for the comments from the anonymous person without fear of reprisal, and openly explain how they verified the legitimacy of the comment author.

Certain contacts would probably end up being used by multiple anonymous commenters. For example, Amnesty International might serve as a contact for people who want to write anonymously about their government's human rights violations.

People who need anonymity for criminal activity or for disinformation campaigns would have a harder time finding a contact to vouch for them that their targets would trust.

If the government finding out who you are would result in imprisonment or death, would you want to entrust your safety to Amnesty International's opsec practices? Against a state actor?

State attackers can produce as many valid identification documents as they need.

I'm sure it is related to the fact they can report N new users to shareholders. I can't see what else explains their complete lack of dealing with obvious fake/spam accounts.

I'd challenge your entire premise that it's easy to create those accounts.

The market price of low-tier bulk-created Twitter accounts seems to be $160/1k accounts. That's not bad. It's in line with services like Linkedin ($175), Hotmail ($150), Yahoo ($120) and Facebook ($150). The only large provider that does significantly better is Google ($300).

That seems pretty easy to me. I'm not sure what premise you think are are challenging by posting prices per account that are under 20 cents each....

The part where @latchkey thought Twitter was doing a bad job at policing bulk account creation, and @samdoidge though it was obviously because they want those bulk accounts. In fact Twitter appears to be doing about as well as most of their peers in preventing bulk creation.

20 cents a pop for a process that should take 10 seconds and should in theory be easy to automate and parallelize for anyone with even basic programming skills? If it were actually easy, that'd be a goldmine.

You seem to have a very different definition of easy than I do.

That Twitter does an equivalent job of policing bulk account creation as their peers just means that it is also just as easy to create accounts at Twitter's peers.

Do these account creations require to supply a mobile phone number?

With [redacted subset] it’s possible to use an account creation path that doesn’t trigger a phone number requirement. A single misstep or threshold crossed, though, and even that path will trigger the requirement.

> [redacted subset]

why?

I clicked on their 'Platform manipulation' link to find out what they mean by that phrase. Early on it says:

"We want Twitter to be a place where people can make human connections, find reliable information, and express themselves freely and safely."

I don't see how bulk-created accounts are consistent with that 'mission' statement. What kind of 'human connection' might result from a bulk account someone paid Twitter to create? How 'reliable' might info from a bulk account be? Why else would someone create a bulk account, except to manipulate people?

Appears they're blind to their own hypocrisy? Further on, at the top of their manipulations list they mention "commercially-motivated spam, that typically aims to drive traffic or attention from a conversation on Twitter to ..."

Yeah. Uh, the Twitter feed contains a constant gush of paid-for spam that is intended to do exactly that.

I think you misunderstand. You cannot pay twitter (or facebook, google, or any other platform) to create accounts in bulk. that's not a service they provide.

Spammers sell accounts in bulk that they have illicitly created or gained access to. Measuring how much those "bulk accounts" cost relative to accounts on other similarly desirable platforms is an effective way to measure how effective a platform is at rooting out spammers and preventing fake & bulk-purchased accounts.

It's easy to misunderstand; that was exactly my point.

The statement about "commercially-motivated spam, that typically aims to drive traffic or attention from a conversation on Twitter to ..." applies perfectly to all of the paid-for advertising. Ever have a problem on Twitter finding your threads because of the 'commercially-motivated spam' of paid-for ads thickly laid on? Ever miss seeing a message someone left?

The ads obviously and continuously break up the conversation between human beings to encourage them to visit promotional sites. The difference between paid and unpaid spam is an economical one for Twitter; for the user its a deluge of distractions.

You could argue that the paid-for commercial advertising is 'saner' or 'nicer' or 'less damaging' than the agenda-pushing bulk accounts. But in the end, both are 'manipulating the platform'. 'Making human connections' has become an ancillary goal.

Unhappy with the verdict? (Down-voting is Sooooooooo easy.) Read all of this. I am not alone.

https://news.ycombinator.com/item?id=21848468

The Dude is my shephard, I shall not want.

Where do you find information like this?

What's wrong with creating fake accounts? Nobody has to listen to what they say. This worry about disinformation seems to be the idea that people see multiple accounts making some claim and wrongly conclude that it represents a popular opinion, and so believe that claim themselves. But that's a ridiculous way to form beliefs even without fake accounts. Just the selection bias of people who would use a particular service already makes it non-representative, let alone whatever algorithm the service uses to decide what messages to show you. And of course popularity doesn't make a claim true anyway, even if you do get a good sample.

What if they're not even fake accounts, but some big group (like a country) just organizes its members to post particular things? Should those users be banned? Or a business pays people to promote their products (the business model of influencers)? What if that organizing isn't particularly formal but is gently guided in one direction in the form of propaganda or advertising (nationalists and fans)? Now do we have to ban all users from some country because they've been too brainwashed by their local propaganda to be trusted as acting like independent human beings? Ban all Christians? They seem to keep repeating the same false information like robots, and somebody might get fooled into believing it.

Q: "What's wrong with creating fake accounts?"

A: signal:noise ratio gets worse when there are tons of noisy bots spamming everywhere.

Q: "that's a ridiculous way to form beliefs"

A: Yeah, but that IS how the (vast?) majority of society changes their beliefs so it does matter.

Q: all your other implied questions

A: it's against the ToS. People who don't like that fact are SoL.

> on behalf of — but not necessarily with the knowledge of — their clients

I get that twitter has to cover its assets, but how likely is 5000 troll strong army working on your behalf, as vocally as possible, and as a country with heavily monitored internet, not notice. The razor of fuzzy logic falls in the obvious place.

The 5000 is just the sample in the dataset, taken from the full 88000 accounts that were part of this operation.

I hate that you have to have a Google account to access/download this data.

> Dec 2019 Saudi Arabie - Media (1.3 TB)

.. thats a ridiculous amount.

I love the new oxymoronic phrase: "aggressive liking". I had to first get past the hilarious opener: "Transparency is at the heart of everything we do at Twitter."

To be honest Twitter are WAY down on the list of offenders.. Case in point are articles like this

"aggressive liking" is that like a "heart attack"?

I think most HN'ers can appreciate the idea that the way we read technical news articles is distinctly different from how non-technical people (say, your Grandma) would read them. Two different parties can read the exact same article, but come away with very different interpretations. (Some of the reasons for this are obvious, but depending on the topic there are also others that are not so obvious.)

In the spirit of cultivating understanding of the differences between individual people with the goal of working towards a more harmonious, peaceful future, and also just for a little tongue-in-cheek fun on a Friday morning, here is a bit of insight into how (eye-catching key phrases excerpted) a distrustful-of-all-authority, possibly insane, unapologetic conspiracy theorist hyper-critically reads articles like this...

> Transparency is at the heart of everything we do at Twitter.

"transparency", "everything"

> That’s why we routinely disclose datasets of information operations we can reliably link to state actors.

"That’s why", "reliably", "link to"

> Today, we are sharing comprehensive data about 5,929 accounts which we have removed for violating our platform manipulation policies.

"Comprehensive"

> Rigorous investigations by our Site Integrity team have allowed us to attribute these accounts to a significant state-backed information operation on Twitter originating in Saudi Arabia.

I wonder what the terms "investigations", "attribute", and "originating in Saudi Arabia" mean, precisely? Is source code available so we can see for ourselves what they actually mean? What with transparency being at the heart of everything they do at Twitter and all.

> In order to protect the privacy of potentially compromised accounts repurposed to engage in platform manipulation, and in response to researcher feedback requesting that we pre-filter unrelated spam, we have not disclosed data for all 88,000 accounts.

Hmmm.

> In the interest of offering meaningful transparency, the dataset we are disclosing includes a representative, random sample of the fake and spammy accounts associated with this broader network.

"Meaningful" transparency.

"Representative, random" sample.

"associated with"

> Our investigations have traced the source of the coordinated activity to Smaat, a social media marketing and management company based in Saudi Arabia.

"Traced", "the source"

> Our in-house technical indicators show that Smaat appears to have created, purchased, and/or managed these accounts on behalf of — but not necessarily with the knowledge of — their clients.

"technical indicators", "show", "appears to have"

-----------------------

Now, an obvious shortcoming (in addition to the obvious mental health issues) contributing to my ludicrous interpretation of such articles is my lack of knowledge in current, (known) state of the art capabilities in spoofing identity on the internet (all criticism welcome), as well as the confidential processes and capabilities within Twitter.

For the latter, there is some PR-speak information here:

https://blog.twitter.com/en_us/topics/company/2019/informati...

> How do we do it?

> Our Site Integrity team is dedicated to identifying and investigating suspected platform manipulation on Twitter, including potential state-backed activity. In partnership with teams across the company, we employ a range of open-source and proprietary signals and tools to identify when attempted coordinated manipulation may be taking place, as well as the actors responsible for it. We also partner closely with governments, law enforcement, and our peer companies to improve our understanding of the actors involved in information operations and develop a holistic strategy for addressing them.

"identifying", "suspected", "potential", "a range", "proprietary", "signals", "identify", "may be", "actors responsible", "improve our understanding"

-----------------------

Possibly related:

https://toinformistoinfluence.com/2012/06/15/images-a-matter...

https://en.wikipedia.org/wiki/Epistemology

https://www.epsilontheory.com/gell-mann-amnesia/

https://en.wikipedia.org/wiki/IP_address_spoofing

https://theintercept.com/2017/02/20/how-to-run-a-rogue-gover...

https://www.expressvpn.com/support/troubleshooting/spoof-loc...

https://en.wikipedia.org/wiki/Edward_Snowden

https://www.reddit.com/r/conspiracy/comments/4g7jay/updated_...

-----------------------

Meta: How a non-conspiracy theorist might read this comment:

"So what you're saying is", you believe that this article, and all articles like it, are 100% fake news, that everything is a vast conspiracy.

Possibly related:

https://reformedperspective.ca/so-what-youre-saying-is/

https://en.wikipedia.org/wiki/Logic

https://en.wikipedia.org/wiki/List_of_fallacies

https://en.wikipedia.org/wiki/Cognitive_distortion

https://medium.com/datadriveninvestor/we-human-machines-hall...

I don't believe this can be ever solved. I think the best thing that can be done is wait 1-2 generations; by then nobody will believe anything they see in the media or in social networks.

Do you think that future world where nobody knows anything will be better than the current world?

Knowledge inequality is growing the same way as economic inequality is growing.

Here's a difference view - https://www.edge.org/response-detail/10464

And Twitter themselves are talking out being "excited" about the emergence of "social intelligence" - https://blog.twitter.com/en_us/topics/insights/2019/the-stat...

> I think the best thing that can be done is wait 1-2 generations; by then nobody will believe anything they see in the media or in social networks.

That's not the best thing, that's the worst thing. The goal of disinformation campaigns is to make people so distrustful and confused that the adversary's social cohesion is compromised to the point where they become disorganized and less able to respond to threats.

This New York Times video explains it very well: https://www.youtube.com/watch?v=tR_6dibpDfo.

Yet nothing about the American spam accounts doing the same thing in American politics.

I was also curious about this as when I first looked at the link it said 'state-backed' which immediately put the NSA and the likes in my mind. Then it went into Saudi propaganda. But if we look at even recent HN headlines with the likes of Facebook [1] we can see that political misinformation is a big problem in the US as well and I wonder why this article focuses on the Saudis and not state backed information operations as a general world wide problem. My guess is that people don't consider your average politician buying fake news ads to swing an election to be 'state-backed' where as if the president was caught they would.

[1] https://thehill.com/policy/technology/468216-zuckerberg-doub...

> ...when I first looked at the link it said 'state-backed' which immediately put the NSA and the likes in my mind. Then it went into Saudi propaganda.

> ...I wonder why this article focuses on the Saudis and not state backed information operations as a general world wide problem.

This is an announcement of a specific disclosure of information related to a state-backed influence network from Saudi Arabia. Not all statements and articles about state-backed information operations need to be general, and it's actually very helpful to have ones that focus on specific instances.

Twitter is a US based company. We will never see reports regarding CIA or NSA activity because those agencies can simply give Twitter a gag order.

Also, the CIA may prefer other techniques than spamming social media. There's no good reason to think they're following the same playbook as the FSB.

With what we learned about the NYC FBI and from the recent IG report, why would we not assume that there are political factions within the intelligence agencies abusing their training and tools to try to help a particular candidates?

> why would we not assume that there are political factions within the intelligence agencies abusing their training and tools to try to help a particular candidates?

That has nothing to do with what I said, and is in fact completely orthogonal to it.

"Transparency is at the heart of everything we do at Twitter."

https://transparency.twitter.com/en/information-operations.h...

"Enter your email to access the archives:"

Transparency by putting up roadblocks to access?!

What's "transparent" about showing subjectively flagged posts? Even using words like 'disclosures' comes across quite fishy considering these are posts marked on belief and some subjectively developed algorithms.

That's not what they're doing. Very clear these aren't just "flagged posts" and that twitter evaluated where they came from.

> Our investigations have traced the source of the coordinated activity to Smaat, a social media marketing and management company based in Saudi Arabia.

Reply on news.ycombinator.com