How many bots are on Twitter? Question is tough to answer and misses the point | Hacker News Reader

How many bots are on Twitter? Question is tough to answer and misses the point

niemanlab.org

30 pointsthm43 comments

The Botometer guys need to give this up. They've been doing this for years yet their models lack credibility, having been subject in the past to 50%+ false positive rates which were never acknowledged. Instead they silently hard-coded their websites to fix the particular examples being used to prove FPs (their methodologies are never replicable, so a black box API is all you get).

When you sit down and read these papers you find the whole field of academic bot research is just ideologically motivated quackery. It isn't taken seriously by anyone who actually works in the spam fighting industry, but does harm the overall credibility of science. If these people actually want to work on bot fighting (like I did) they need to quit academia and go apply for jobs at tech firms. Anything they try produce externally will be noise, because as they acknowledge:

"External researchers do not have access to the same data as Twitter, such as IP addresses and phone numbers. This hinders the public’s ability to identify inauthentic accounts."

It doesn't hinder it, the lack of such basic signals makes it impossible, which is one of several reasons their studies keep yielding junk-quality results.

Further reading:

https://blog.plan99.net/fake-science-part-ii-bots-that-are-n...

https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f...

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3814191

https://archive.org/details/hopeconf2020/20200726_2000_Peopl...

Very useless article. "It's hard to count them, so let's just acknowledge the difficulty of the problem instead of solving it."

I was puzzled why the premise of the controversial headline was so flimsy, until I noticed halfway down the page that I was being sold a product.

Yeah clickbait. The issue with bots comes down to the valuation of twitter end of story. Musk wants to bring that valuation down and he's using bots as a method to do that.

To be honest he does have a point (I hate this) as I could easily create and app with 500 bot users and 50 real users. Selling it for if I had 550 is really scummy.

Is ironic because one of the reasons he public stated to buy Twitter, was to end with the bots.

It would be useful, in so many ways, when there would be a way to prove that you are a unique human being without revealing too much of your identity. An intermediary NGO could provide that, if they in turn integrate with national identity registrations (many countries have these for tax, healthcare and social security registrations. As a policy, it should only provide proof of existence, nationality and “over 18” as data points.

It can help to solve problems with bots, account theft, as well as accountability for hate speech/intimidation (the system should allow law enforcement to reveal identities over which they have jurisdiction).

Then you would just get something like this [1]. As long as the rewards from spam outweigh the cost of spam, people will try to spam. What I don't understand is that Google solved spam detection 10 years ago, yet Twitter somehow can't solve it.

[1] https://www.youtube.com/watch?v=Y-W0CBOGnnI

I really wouldn't say that Google has solved spam detection. If you search for any kind of purchasable consumer product, the first page of results mostly contains spam blog entries of very low quality. Most of them are "10 best X" or, even more useless, "50 best X" lists with the advertised product(s) on top of the list. Some of them are clearly auto-generated and still often make it to the top 20 results. That's why many people including me nowadays search for "search term +reddit" when looking for product information.

If you include Youtube under the label Google (i.e., Alphabet), it's even worse, there is tons of spam and deliberate political misinformation in the comment sections, although arguably most of these are probably generated by humans. For example, any TV report about Ukraine on Youtube is spammed with pro-Russian troll posts from St. Petersburg and likely hired posters from India. There is also lots of commercial spam in comments.

So no, I don't think they've solved the problem. They're using captchas against bots and that's about it.

Web spam is a separate problem to bot-driven spam posting on social media, despite using the same word.

Spam is also not the same thing as posts you disagree with. I went to YouTube, searched for [ukraine war report] and picked the first result that came from TV news to try and check your claim [1]. The top comments are all clearly human, some are claiming to be Ukrainians, and are all anti-Russian.

Regardless, your post is a good illustration of the problems with academic bot research. Although researchers claim to be researching spam bots, their methodologies are often defining any post they don't agree with politically as "spam" or from "bots".

[1] https://www.youtube.com/watch?v=nx-6Y00MrFo

No, when I use the word "spam" I do not mean "posts I disagree with", I was referring to posts that are deliberately intended to misinform, are repetitive, or are intended to lure people somewhere else, and are posted by people who are paid to do that or by bots. It is false to assume that these type of posts and associated accounts cannot be detected or do not exist.

I agree that spam by bots differs from spam by humans. However, they need to be dealt with in union because they usually go hand in hand, i.e., bot nets often support and amplify select human posts and the human origins of these networks can be mapped and traced back to particular actors and sources.

I doubt anybody would disagree with your general claim that there can be methodological problems with detecting any kind of spam, whether by bots or by humans. Of course, that's difficult, especially for researchers with only limited or no access to sensitive account information such as creation date, post history, IP numbers. If it was easy, then the spam problem would have been solved already, but I have argued that Google hasn't solved it.

"I was referring to posts that are deliberately intended to misinform"

But how do you know they're misinformed/intending to misinform, unless you disagree with the post?

First of all, I took you to use "disagree" in the sense of politically disagreeing. You can identify alleged information as being false based on other, more reliable evidence independently of whether you agree or disagree with any political position it is used to support. Second, often the original source can be identified, and in certain cases it's clear when an original source is compromised or unreliable. For instance, sometimes the original source are mere bloggers or activists with no privileged access to information. Other times the source is a known misinformation outlet. For example, wareonfakes.com is registered in Moscow, has no identifiable sources or posters or anyone else taking responsibility, and is obviously not upholding any journalistic standards. Third, it is often forgotten that misinformation can also involve correct information - this is even the standard. The misinformation part is in skewing someone's perception of reality. This can be recognized with a bit of common sense. For example, reports of small, anecdotal incidents are irrelevant for judging anything. The "the lady who defied a soldier from country X" were spread both by Ukraine and Russia, and are irrelevant to evaluating the overall situation. Other reports are not credible from the start. A typical example was the "Ghost of Kiev" story, which doesn't even need fact-checking to tell it's main purpose was propaganda and the informational value was almost zero. Fourth, not all of this is hard to recognize, sometimes misinformation is spam in the traditional sense. For example, there used to be (or still are) a massive number of posts under Youtube comments to advocate the not-yet-closed Youtube channel of a vlogger of US origin who claims to be a journalist (for which the person has no credentials) and is embedded into the Russian military campaign. These kind of repetitive referrals to other outlets are spam under any definition of "spam", and violate Youtube's ToS. Fifth, you can recognize intentions by analyzing the texts and behavior of the actors, and analyzing their amplification networks (who repeats which Tweet, for example). Humans are fairly good at recognizing bad intentions, although it's arguably harder when you only have written text available. A social network analysis helps, since bad faith actors elicit different publication patterns than ordinary users.

Last but not least, in the case of Russia's current aggressive war against Ukraine, it's not just about misinformation. Not everyone is aware of that, but in many jurisdictions supporting Putin's war can constitute a crime punishable under penal law, just like using the word "war" can constitute a crime in Russia. In my opinion, all posts in favor of Russia's aggression should at least be removed from social media. This shouldn't even be controversial, 141 countries have condemned this aggression which violates all international laws. It was not common during WW2 for US companies to give the German Nazi party free airtime on all broadcast channels. People could have their personal, pro-Nazi opinions in allied countries, but expressing them publicly had - and should have had - negative consequences. There is no reason to think this should be otherwise in the current conflict, which is essentially a large proxy war between NATO countries + Ukraine as defenders against the sole aggressor Russia. Not just misinformation should be stopped in this particular case, but also open expressions of support for Russia, displaying the swastika-like "Z" sign, and so forth. And, of course, the NSA and other intelligence agencies should hack back to destroy Russian propaganda outlets and take them off the net.

However, I admit that this last point kind of deviates from the original discussion. In a nutshell, there are many indicators for spam and many indicators for misinformation, and you judge these in a way similar to how a judge would evaluate cases that primarily involve circumstantial evidence. Humans can do this reasonably well, algorithms are still bad at it.

Isn't the number of bots primarily relevant to advertisers? This article entirely misses why the question is even worth asking.

The number of bots misses the point for advertising too. If a company spends $X on Twitter ads and measures a boost of $Y in sales from Twitter leads, how many humans vs bots viewed the ads is neither here nor there.

That said, the question is certainly worth asking beyond the realm of advertising. Media has a significant impact on people and society beyond its profitability.

The whole point is that most of Twitter isn't cost per action ads like facebook, but brand advertisers who probably are just looking at impressions.

If those impressions are off by X% due to bots, that's a problem. Especially if as a public company you've declared that issue at 5%, and it's more.

"Half of my advertising budget is wasted. If only I knew which half". Not so easy to know sometimes

It depends on whether you're paying per impression or per conversion.

If you pay an advertising company a rate per view and the advertising company charges you knowing that a large percentage of those "views" were fake then you're being ripped off.

Well, that depends. If I pay X per 1k views what I care about is how much I sell as a result of that. If I sell a lot and profit would I care that some huge percentage of my views were bots? Conversely, if I sell very little would I care that there were zero bots?

If you are advertising just to get an idea out, or for something you have no capacity to measure, sure. You need to know accurate numbers. But if you can track conversions on your end, and conversions are good, does the exact number of real views really matter?

Of course it matters if you’re being billed by the impression! No matter how good your conversions might be, your margin would be unquestionably higher without the fraudulent impressions.

not necessarily. It really comes down to how the market sets the price per impression. If the impression cost is ultimately set by conversions, removing the bots will simply raise the price for impressions and you end where you started.

Most Twitter ads are brand advertising, not performance based. And it’s harder than you think to even track conversions from Twitter ads (because of cookies/incognito mode/lack of referral data from some browsers).

As far as I can tell the claims revolve around mind control bots not advertising viewing bots.

Nieman misses the mark here. They are worried about some other fight - "authentic" vs "inauthentic". That's fine but not the issue at question.

I picked a random news org. and scrolled through their feed. It was The New York Times with 53 million followers. Scroll through their last 100 tweets or so and look at the engagement. 5-50 comments, 20-200 retweets. I used to get that level of engagement back in the day when I used Twitter with a couple thousand followers. Something’s off.

Note. I’m not American and have no political affiliation to have selected NYT.

EDIT: this was before the shooting that's just happened.

Twitter (used to?) recommend accounts to follow (I remember Obama, Shakira, WaPo, and I'm sure NYT) when you sign up with a new account, and those accounts probably have highly inflated number of followers with extremely low engagement (new users don't always stock around).

Recently created an alt account and they still do this.

It was mostly “influencers”.

One thing I have noticed is that Twitter REALLY wants me to follow Elon Musk. He’s at the top of pretty much every recommendation I’ve seen on the site.

Who comments or retweets news headlines? There is nobody on the other side of the account who is going to engage with you - the accounts are literally bots that publish the headlines from their feed.

Lots of people follow the news / latest most followed celebrity account and dont engage with that person at all.

apparently that's "normal" for New York Times in particular

check out the views to comments ratio on this https://www.youtube.com/watch?v=K4II9cYrQvE

"...it is important to be clear here that Musk is lying. The spam bots are not why he is backing away from the deal, as you can tell from the fact that the spam bots are why he did the deal. He has produced no evidence at all that Twitter’s estimates are wrong, and certainly not that they are materially wrong or made in bad faith. (Musk can only get out of the deal if Twitter's filings are wrong in a way that would cause a 'material adverse effect' on Twitter, which is vanishingly unlikely.) His own supposed methodology for counting spam bots is laughable. Yesterday Twitter’s chief executive officer, Parag Agrawal, tweeted a thread explaining in general terms how Twitter estimates that fake accounts represent fewer than 5% of its count of active users, and how this analysis can’t be easily replicated by outsiders (because they don’t know which accounts are real, and also because they don’t know which accounts Twitter counts as daily active users). It seems clear that Agrawal’s thoughtful answer is basically correct. Musk responded with a poop emoji.

More important, nothing has changed about the bot problem since Musk signed the merger agreement. Twitter has published the same qualified estimate — that fewer than 5% of monetizable accounts are fake — for the last eight years. Musk knew those estimates, and declined to do any nonpublic due diligence before signing the merger agreement. He knew about the spam bot problem before signing the merger agreement, as we know because he talked about it constantly, including while announcing the merger agreement. If he didn’t want to buy Twitter because there are spam bots, he should not have signed a contract to buy Twitter. No new information has come to light about spam bots in the last three weeks.

What has happened in the last three weeks? Well, the prices of tech stocks have gone down, making the $54.20 price that Musk agreed to look a bit rich. (Snap Inc., a social-media competitor to Twitter, is down more than 30% since Musk made his offer on April 13.) And the price of Tesla Inc. stock, which he is relying on to finance part of the purchase price, has also gone down, making him poorer and making the $54.20 price look even more expensive. (Tesla is down almost 30% since he made his offer.) So he is angling to reprice the deal for straightforward market reasons. But that is very clearly not allowed by the merger agreement that he signed: Public-company merger agreements allocate broad market risk to the buyer, and he can’t get out just because stocks went down.

So he is pretending that he wants to reprice the deal for other reasons. He is not pretending very hard — the poop emoji is not going to hold up in court! — but he’s doing enough to confuse the public and give his fans a pretext to believe that he is really the victim here."

Twitter's Board are a clown car.

[1] https://www.bloomberg.com/opinion/articles/2022-05-17/elon-m...

I'm not sure if it's appropriate to paste such a large citation as a comment here. You could have just said "Read Matt Levine's article" and then added your own thoughts.

Yeah that weirded me out. I knew I'd read this take somewhere before, but it wasn't clear that I got to the bottom that the OP had just copied it without any of their own commentary.

I’ll allow it.

The author Matt Levin really doesn't like "Le Bad Billionaire Man" and posts against him frequently. This piece reads with a clownish amount of bias.

I think the piece was fairly well written, though you can tell he doesn’t have a great deal of respect for him. Musk has honestly said and done a fair amount of clownish things, so even if you wrote an article merely listing them with no commentary it would look ridiculous

Regardless of bias, anyone who interacted with Twitter in the past 10 years must have known about the bot problem. Complaining about it after you've signed the deal just seems suspicious. It's not as if Musk is a very trustworthy figure either

Musk said the large number of bots was why he was buying twitter in the first place. If they can prove that the number of bots is less than the 40-90% that Musk claims will he pay extra because it is a better asset than he thought?

So I forget: Are masses of Russian bots spewing disinformation still swaying public opinion on important issues? Or has there been acknowledgement that that never really happened?

Are bots not entitled to free speech as well?

If you can't distinguish between a bot and a human, is it a bot?

If you can't distinguish between a human and a bot, is it a human?

""" The focus of the recent debate on estimating the number of Twitter bots oversimplifies the issue and misses the point of quantifying the harm of online abuse and manipulation by inauthentic accounts. """ This article is missing the point. The reason Musk wants to know how many bots there are is because a lot of the advertising revenue isn't "pay per click" ads, but just companies advertising for more eyes to see it. In that case, companies aren't going to pay if it's not real human eyes seeing it

bit of clickbait

> The reason Musk wants to know

Musk wants to know so he doesn't have to pay the price he agreed to pay given the entire tech stock market just took like a 50% hit. It's nothing to do with bots and everything to do with getting out of the deal/getting a more sane price and honestly fair enough I wouldn't want to massively overpay for something like that either.

The point is: Twitter is useful for who?

For users I see nothing useful. For third party using users tweets to get live coverage of something a little use.

For human network and mud analysis by Twitter owners and those who buy/commission such studies a significant use.

So the point is: WHY THE HELL so many people want to work for free mostly against their own interests spending time on the platform? That's an interesting point because having flock of humans (sorry for being so rude, but that's is) willing to work for free for someone else interests is VERY interesting... I have many things I'd like to outsource for free to third parties willing to work gratis for me. Just basic stuff eh! Who want to came at my home to clean it up regularly for free or even pay for being allowed to clean up more regularly?

Reply on news.ycombinator.com