Hacker News Reader

Comment by krona | Hacker News Reader

I just ran my own Twitter account through the so called 'Botometer' used to form the conclusion of the article. Apparently I'm 4.7/5 bot.

Needless to say I didn't finish reading the article.

View on HN

I ran my account through it and got a 0.0 rating, and ran it against my followers, and got similarly low scores for the accounts that I know to be real people (the highest score for a follower I know to be a person was 0.8). I saw higher scores for either accounts that followed me out of nowhere (I'm not interesting enough to draw real strangers), or for accounts that are run as public profiles (an artist with a public account for their comics). I'm inclined to assume that you likely have either atypical or impersonal twitter habits.

Or maybe that's just buggy software.

It's not buggy: it detects patterns like the majority of posts being retweets instead of new posts; any new post consisting of an image and accompanying text instead of a message or thread; replies being short comments on top responses to popular posts; and a high degree of connectivity to other accounts that are likely bots.

False positives are generally curated accounts, corporate accounts, and government accounts (which are really bots with their software in managerial documents); porn accounts; and people who reblog the news but don't really "use" twitter.

If it detects my account as a bot I would say it's buggy, because I've done none of that stuff.

Or you just post a lot of misinformation.

Right, yes, that must be it.

When I was in the spam blocking business we joked about the problem of the Russian urologist. Somewhere, there must be someone who really is trying to buy Viagra from a Russian e-commerce site. But the other 99.99999% of the time it's spam. Perhaps this person's Twitter suffers a similar fate: all the classification features align with robot traffic, but they aren't a robot.

That would nullify the study entirely since the criteria in the test directly overlaps the criteria of the results.

There are multiple people in this thread that have been incorrectly tagged as bots, I’d like to see proper tests done with the software identifying the level of accuracy. It shouldn’t be too much to ask since their entire study is hinged on the accuracy of this software.

And if it's anything like some other naïve bot detection algorithms I've seen, it might be viciously biased against non-English-speakers. To the extent of, say, labelling anyone primarily using Arabic or Cyrillic alphabets as a bot.

Or that writes a lot in Dutch.

Just the other day a prominent dutch account got auto-blocked on twitter for saying 'Die Rutte toch.', where 'Die' (a pretty common dutch word, which in this case translates as 'That') was mistaken for a threat to our prime minister.

You may need to test for confirmation bias. I am sure you and people you know are well behaved and courteous on twitter and social media in general, and so would not trigger any alarms. Have you tried the bot-test on more contentious accounts that you know to be human?

I just checked a few of the more abrasive people I know, both of whom also do a lot of non-quote Retweets, and both also scored quite low (0.4 and 0.1).

If only this was 2 weeks ago, we would know a certain contentious presidents' account to bot tests... but as of now, why follow anyone who is not well behaved?

Have you ever injured a human being or, through inaction, allowed a human being to be harmed?

While walking along in desert sand, you suddenly look down and see a tortoise crawling toward you. You reach down and flip it over onto its back. The tortoise lies there, its belly baking in the hot sun, beating its legs, trying to turn itself over, but it cannot do so without your help. You are not helping. Why?

The article is distinctly Orwellian. I am a climate skeptic and have participated in such discussions, and found that my bot-rating is very high. Other people whom I know by reputation and online presence elsewhere are also categorized as not being actual humans.

Along with this, there is the pre-emptive and dogmatic assertion without proof or serious argument that any disagreement with official dogma must necessary be deliberate 'disinformation' spread by those with an immoral vested interest in spreading falsehood.

So the correct conclusion is not necessarily that most climate change skeptics are bots, but rather that climate change skeptics are likely to be characterized by the software as bots. I wonder what else makes you more likely to be characterized as a bot?

>So the correct conclusion is not necessarily that most climate change skeptics are bots, but rather that climate change skeptics are likely to be characterized by the software as bots.

Yes, that is as much as we can deduce from the data presented in the paper, given that they apparently just took the 'Botometer' verdicts at face value without taking the extra steps to see if they were indeed accurate. It may, of course, be absolutely true that the 'denier' accounts are just scripts, but further investigation is necessary to establish this.

> I wonder what else makes you more likely to be characterized as a bot?

That would best be established by examining the bot-detector's algorithm. Another commenter on this thread noted that he received a high rating based on the fact that he rarely logged into his account.

So now we'll see twitter ban this bot because it "dehumanizes" a large group of people like the NPC meme right?

I listened to the drilled podcast two weeks ago and season 3 was very eye opening to me because they did an in depth look at how corporate PR has evolved to influence public opinion in very subtle and powerful ways. We’re talking oil companies setting up contracts the size of $25 million to influence opinion on single issues and they work with psychologists to take advantage of the shortcomings inherent in our minds.

The result is that in 2021 people tell me things that influence their opinions that are either misleading, out of date or absolutely false. We’re talking things like climategate or that volcanoes give off more co2 than all of human civilization.

I’m just a regular person, I don’t believe in climate change, I’ve researched it, it’s something I’m concerned about. A million dollar PR campaign against me and my friends, family and coworkers is not something anyone can standup against.

The dogma that you perceive comes from people reacting to dissatisfaction from an important issue that isn’t being taken seriously. We are citizens, regular people, against what amounts to artificial intelligences that are maximizing the number of paper clips they produce. David and Goliath doesn’t come close to the situation we’re up against.

How on earth do you come to that conclusion? Personally the physics of climate change is kind of a basic back of the envelope calculation that should be obvious to most educated people.

So I genuinely wonder how people can look into it and not look at the absorption spectra of CO2 go ahh I see and then move on. I mean such a calculation isn’t enough for policy changes or anything, but it’s putting a high hurdle on the other side.

Is it some disagreement about the physics involved, or just a bunch of people you believe saying something different, or what?

The dispute is typically over the practical implications of it, not the pure physics of "the average global temperature will rise as long as this trend continues". And this is a topic where disinformation comes from both sides - I pretty regularly run into people who've been convinced that climate change means the planet will become uninhabitable.

There are 2 types of people. Those who think in statics and those who think in dynamics. Climate change is not basic back of the envelope calculation specifically because we're dealing with a dynamic system. The more complex the world, the less you can grasp it from academic studies.

In what way is that complexity meaningful? If I put a pot of water on the stove the specific way the water swirls around is a chaotic non linear system, but it’s not going to keep the water from warming up.

Similarly, if hypothetically increased global temperature results in increased cloud formation that reduces the impact of climate change that presupposes some increase in temperature otherwise the change in cloud formation doesn’t happen. As such it’s reasonable to argue about the magnitude of change but not the direction of change.

I can clearly see arguing about a specific location bucking global trends because there complexity actually plays a direct role, thus the term climate change. But, globally I can’t see how everywhere on average would buck these trends.

That complexity is meaningful as it precludes precise modelling. Now that does not mean we should not curb CO2 emissions, rather it is the degree of opacity and uncertainty in a system, as well as asymmetry in effect, rather than specific model predictions, that should drive the precautionary measures.

Push a complex system too far and it will not come back. The popular belief that uncertainty undermines the case for taking seriously the "climate crisis" that scientists tell us we face is the opposite of the truth. Properly understood, as driving the case for precaution, uncertainty radically underscores that case, and may even constitute it.

It seems obvious. Just tell the facts!

You may have great social circles with intelligent and curious people, but in my life the facts don’t work because the facts don’t matter. The number of people actually interested in the science and not just the science (tm) is dismally low, we’re talking less than 10% in my experience. This includes people who have degrees in physics and engineering and a university engineering professor I knew.

People are hungry, vulnerable, emotional, scared, and prideful creatures. That’s what we are; and we like stories. We like stories where we’re hero’s on an epic journey with villains to defeat.

If you tell someone a fact that breaks their story, makes them scared, and makes them less prideful, you’re not having a conversation for very long, doesn’t matter if you can support it with evidence and reasoning. What PR added on top of this human tendency is people think about climate way more than they otherwise would, they think about it in terms of a story, in us vs. them terms (PRs narrative), and if you say something counter to that narrative, now you’re the villain.

I’ve had people I respect flat out tell me that climate change is a hoax and if we reduced carbon emissions it would just be virtue signaling. They are not joking, they are serious and they’ve been told this by talking heads for over 20 years.

You’ll never get to the back of the envelope calculation.

Did you have a typo in your earlier message "I don’t believe in climate change", or is there some angle or point in that I'm missing?

I suspect it was meant to be read as “I don’t [just] believe in climate change, I’ve researched it, it’s something I’m concerned about.”

I also have doubts that the climate exists.

Don't feel too bad, I'm 3.9/5. Must go in for service now.

I have never had a Twitter account, so unfortunately I cannot do the same. I'm hoping more people here do the same as you so we can get a cross section of the accuracy for that specific bot detector.

I have problems with articles on both sides of the issue using all kinds of speculation and innuendo to try and discredit the opposite side. I would rather just read articles that cover studies explaining results (data) and methods used to support their position.

Who are these opposite sides?

Presumably people who want to stop cooking the planet vs people who are ok with that as long as they make more money?

A ML model giving a single false positive is generally uninteresting and meaningless in evaluating its overall performance. Also, note that the details clearly state that, even for high bot scores, some non-trivial percentage of examples in their training data were still labeled as human.

As an aside, for those interested, the GitHub page for the tool's Python API (https://github.com/IUNetSci/botometer-python) lists relevant papers regarding the functioning/design of the tool.

I ran my account through both Botometer and Bot Sentinel. Got 0.4/5 on Botometer, and 0% on Bot Sentinel.

Kind of annoyed that Botometer requires me to authorize it to access my Twitter account, when all my tweets are public (and who cares about bots that make private tweets, if that's even a thing). Bot Sentinel didn't require this.

https://botometer.osome.iu.edu/

https://botsentinel.com/

I don't know what the Botometer does, but it labeled my clear as hell bot account (automatic posting of started games in a computer game) as human. Also I found no output of applications that were used to send a tweet. Most bots can easily detected by that. Mine also.

Yeah they gave me a 4.1/5 on account of not being active enough. Their method is sketchy for sure.

Curious if you get a similar result with BotSentinel https://botsentinel.com/

NB: not affiliated with Bot Sentinel

I tried both this and the other and I am happy to say I am not a bot on either. I did get a .7 on the other and a 0 on this one.

Interestingly it identified my sister in law as a bot, but looking at her account, I'd assume the same. Anyone else I checked seemed accurate as well.

The Stepford Wives...

What I don't get is why they need to access my account to check public tweets from another account.

To run more Twitter bots, perhaps? (to counteract climate disinformation with the absolute climate truth and science-based conclusions, of course)

Most of these things just use it for API access, from what I’ve gathered. Haven’t looked into this one specifically.

That was interesting: I tried https://botometer.osome.iu.edu/ , and got a score of 0.2. I then asked it to look at my followers and it spotted several obvious spam followers, and correctly identified some "quote of the day" accounts as bots.

I only get a 1/5 and almost all my followers are bots or abandoned accounts I paid for years ago.

Same things with BotSentinel - it's just garbage.

But... you are not a bot. So the result is arguably right?

If it was really good, then yes it's technically correct. But I suspect it just has no idea.

What kinds of things do you post about?

As far as I can tell the score is based on levels and patterns of activity, and whether 'bots' follow you. Not the actual contents of the tweets.

Interesting. Nobody in my network scores over 2.0, mostly not over 1.0.

I follow a (very) wide spectrum of accounts on Twitter and the following seemed true for the accounts I follow which scored highly likely to be bots:

- it's 50/50 accurate picking actual bots I know I follow

- the more simple the reading level, the higher it scored

- however, extremely heady accounts also score high

- the farther right/left the account goes seems to up the score a little

- accounts that quote authors frequently score very high, even if they're human behind them

- everything from Chinese state media scored higher than anything

- everything that was frequently highly critical of China scored extremely high

- most Trumpers and Antifa I follow had very low scores (but most are high-up/origin accounts)

- most news organizations from both sides scored middle range, including known propagandists and "reputable" organizations

- accounts from both sides of the COVID discussion were mostly low

Hard to say, but my best guess is "low effort" is the defining factor. Only exception is the pro/anti-Chinese stuff, which is definitely a mix of real people and bots.

I keep pretty good tabs on the accounts I follow and I think I'm pretty good at determining who is "a real person with real feelings saying real things they really think" and bots, and from this analysis, I have zero faith that this tool has any modicum of reliability in a research setting.

I think it's possibly accurate enough to say "yes, this is definitely a human", but totally fails to identify bots. Thus, any research that uses this tool as a measure of bot influence is way off, because there are way too many false positives. Which I guess is a good thing for anyone who wants to say, "look at all these people being suckered by all these bots!" which is pretty unscrupulous.

YMMV

Reply on news.ycombinator.com