A peek into Reddit's anti-spam internals | Hacker News Reader

A peek into Reddit's anti-spam internals

lyra.horse

185 pointsOuterVale77 comments

I cannot emphasize strongly enough just how deeply pervasive the spam is at Reddit. I'm a mod at the ecommerce subreddit, and I've only caught some of the AI-powered marketing operations because in one particular campaign that was making fictional claims about things I had direct knowledge about. Once I looked into the post history, and started to untangle the web of accounts that formed a self-supporting community of posters and commenters, just subtle enough to get genuine engagement, but specific enough to make the kind of posts that the LLMs will siphon up and regurgitate.

It's not just shady little operations. I'm speaking specifically about the SCAYLE ecommerce platform, in my example. They've got Zalando money to play with, and as a German platform that's trying to break into the North American market, it appears they've made a bet on indirectly spamming the LLMs with fictional tales of commerce replatforming horror stories. At first, they're some of the more interesting topics in a sea of really useless posts, with contributions from people who seem to have some real experience with enterprise ecommerce. I was a little suspicious, but these interaction campaigns were spread out enough that I didn't put the pieces together for months. Of course, to go back on what I said at the top of the paragraph, maybe SCAYLE is shady, and I'm giving them too much credit.

The good news is, some of the AI powered tools that mods have access to are getting better at surfacing suspicious patterns of behavior. However, I still find I have to manually address these campaigns.

In the cat-and-mouse game with these marketing jerks, I'm always reluctant to surface what's working and what isn't. This is an interesting post, but it's going to make things worse. Ah well.

I remember reading years ago about some corrupt mod in one of the image subreddits - he or his friend had started some image hosting site and had six different Reddit accounts that he used to upvote posts that used his site and downvote all other posts. It took people a long while to notice what he was up to.

And now automate and scale that with Claude/OpenAI/Gemini/whatever. It's insidious and terrible.

Why would you need an LLM to automate that?

the Unidan incident?

iirc it only got noticed at the time because of an argument between him and Ecka6 which led to the somewhat famous "here's the thing you said a jackdaw is a crow" copypasta

Unidan doesn't fit the description. It must have been a different incident.

According to Reddit lore, that was coincidental. The jackdaw/crow thread was simply his last pre-ban comment:

<https://old.reddit.com/r/OutOfTheLoop/comments/2cmdiq/whats_...>

The flip side of this is that for many years it's been basically impossible for a real person to convince Reddit to let them have an account. They track so many signals and if they don't like a single one or a combination, you get shadowbanned - I've tried it a few times since then on different computers on different networks with different email addresses, and I concluded they must have an extremely specific idea of what a new user does and everything else is spamming. For example if I post a few comments within a few hours of signing up, I was always shadowbanned. Because that's what a new user does, you see.

I stopped trying to have a Reddit account in about 2024 when the platform was too obviously enshittified, with no content of any value whatsoever remaining on it.

Once Discord started taking off I was much happier returning to the chatrooms I started with (on IRC), without any algorithm attempting to maximize your engagement. It's much less useful as a resource that grows over time, but the value of something like that has been slashed and burned by LLM's anyway.

In the end the biggest hurdle to getting an account on Reddit at this point is why you'd bother.

For the last several years, I’ve created a new Reddit account, posted heavily, and then deleted it when I felt I needed a social-media break. Rinse and repeat. I’ve never had problems creating those new accounts or having them banned. You just need to verify your signup with some new email address, at some domain that isn’t already a known throwaway-email domain, and accept that some subs won’t show your comments until your account is a couple of weeks old.

But I definitely agree with you that the platform is finished now, even smaller subs that aren’t drawing so much surreptitious spam. The problem is that even if one uses Old Reddit, the vast majority of other posters are using the app. That tends to discourage substantial discussion or community, in favour of daft 140-character shit comments.

When I become a dotcom^H^H^H^H^H^H AI zillionaire, I'll start a web-only platform that blocks mobile user agents. (If you can work around that, you've passed the entry gate.)

I'm sure you love Reddit's decision to allow users to hide their post history.

What I do love is that when you're a mod, that post history isn't necessarily hidden.

I don't think this covers comments (...on subreddits you don't mod)

It's definitely been a lot harder for me to uncover sockpuppetry

I hate that change. arctic-shift.photon-reddit.com still works, at least with comments before the change. Not sure about current comments.

It hasn't been working for about a week now.

rip

The same as youtubes removal of dislikes and more recently they disabled search results with newest results first.

Now I ignore a user based on other criteria (account age post/comment karma + a 50 times compressed repost)

They still let Google crawl, so [site:reddit.com "username"] can be instructive.

I'd be surprised if Reddit wasn't selling tools specifically aimed at seting up and managing operations like that

I would be surprised if Reddit was selling tools like that directly. Rather than just look the other way - because uncovering modern bot operations is a thankless threadmill, and user engagement metrics from fake users are still user engagement metrics.

to be fair, i once accidentally ended up on shreddit or new reddit or whatever they call it nowadays and i think there's something for managing your posts on reddit and seeing analytics about that or whatever

> uncovering modern bot operations

this significantly overestimates how sophisticated the spam waves are compared to like ability. the 80% of spam filtering basically never was really done as far as i can tell.

> a thankless threadmill, and user engagement metrics from fake users are still user engagement metrics.

that's probably it tho

How did you manage to uncover the operation? Is there any things that tipped you off? Is it like, accounts only posting on a precise subset of subs, or how much a network of accounts reply to each other? Or too much linking/references to products, or perfect spelling/identical consistent typos?

I'm curious because it feels like it could be built into a tool to analyse - even if it does become a bit of an arms race.

It's harder now that post history is hidden but I did something similar in the past.

Start with a single comment you think is a shill. Maybe 80% of their post history is vague inane generalities (like "aww so cute" in r/cats, or a reply to a top rated post that only paraphrases the existing context without adding anything new). You can use an LLM to identify every comment or post from that account that mentions a product or service. Take note of everyone who replies to that comment as well as the parent comment. Then use an LLM to identify every post from that original account asking for recommendations (hey r/bidet, what's your favorite bidet), and look at who responds. If you build this graph, draw directional edges based on who replied to who. The accounts with edges both ways across different posts are bots. Rinse and repeat by examing the post history of THOSE accounts. You will end up with a graph with a few loosely connected nodes (maybe false positives) but a tight web of spam accounts that frequently engage with each other.

That's your bot farm. This would be relatively trivial for reddit to implement, if they cared about reducing spam. I got a POC working in a few hours, back before they limited API access

It's a good idea. But this would no longer work now that Reddit hides post history, right? Or does the API still provide a user's post history?

You'd basically need to be pulling all the data for all the subreddits, and then recreate a user's partial post/comment history from that.

It's hidden for users who opt in (not all spambots do, believe it or not) but there are some 3rd party databases that index comments- no idea how complete they are

Ah, I thought it was opt-out and hidden by default now. (For new accounts, that is)

I can't even fathom how mods now handle all this. I was a mod for one of the top subreddits and it was hard back then, a few years before LLMs, and I only had to deal with URL submissions, not comments.

I also help moderate a forum and I noticed the spike in new user signups for spamming has sharply risen over the last two years or so. The majority are most likely using LLMs to automate this. So now we put new registered users into a shadow account where they can post and interact, but it's only visible to them and no one else on the forum for a probation period. It seems to work for now.

Reddit must have some mechanism specifically for non-spamming bots that isn’t covered in this article. I wonder how it works. I imagine the mechanisms are more complex and opaque than anti-spam (with various levels being exposed to the hierarchies of Reddit and government backdoors). These days, I’ve noticed an almost forcing-function that operates to put the minimum spin needed on posts and comments to turn signal to noise. It seems smart enough to not only generate noisy comments but create comments to amplify existing organic noisy comments. I’m sure these systems are decentralized, emergent, and split across numerous nation-states and actors. I’m also fairly certain what we have now is a tenuous balance that has emerged from all these actors and Reddit policing actions as well.

I imagine Reddit has a high-level of insight into this and a certain level of permissibility it grants, both to inflate user counts and to steer public discourse and insight into less productive mean (or productive to certain interest groups at the expense of the people). I think is also an effect that Reddit has become more global and consensus of the USA people is very antagonistic to the consensus of the people of the world so that doesn’t help (+ access to LLMs to make English writing no longer a barrier to entry).

There is some sort of wink wink nudge nudge agreement going on with certain spam accounts. You will see them post article spam with hidden history, and if you look up their posts either via google or any other reddit crawling tool, they are posting all over various subreddits that same article maybe dozens of times. If they comment it is really basic and formulaic and found all over their post histories as well.

I feel like reddit enjoys it as these posts (often political in some way) usually get good engagement which is in line with reddits own incentives for courting advertiser money.

This practice was perfected by gallowboob years ago.

He would spam a link/pic/post and monitor, if the post didn’t gain traction, he would delete and post again as to not trigger protections against the same link being posted.

He was a cancer on Reddit and I’m sure he still exists under different monikers. But now there are 100s of gallowboobs.

> He was a cancer on Reddit and I’m sure he still exists under different monikers. But now there are 100s of gallowboobs.

Yep, for example: If you mention the name "TurboStrider27" on /r/Games, your comment gets shadowbanned.

Holy hell!! The fact that this name still rings out to me years later really says something! And at one point /r/Games was the gaming subreddit for more informed people (compared to /r/gaming). And then one day somebody pointed out a guy named “TurboStrider27” was responsible for a large amount of slop posts. Lo-and-behold, a majority of posts did come from him.

I had no idea this was implicitly permitted (and even supported) but it makes sense if it’s been ongoing for so long. It’s no accident.

gallowboob in particular was an interesting case because he was very much a real person. and oh hell did a subreddit I mod know that way all too well

i think the guy had a like a keyword alert on his username because like one of my co-mods on a subreddit would talk about the guy and then we'd get reports for "It's targeted harassment against me" (which are reports that are sent to the admins) like a few hours later. much to the dismay of him, we had a chat with the admins later and it was like "as long as you're not saying to do vote manipulate or harass the guy it's fine."

i think a lot of it came from the fact that so like if you're modding a subreddit, a lot of people spend their time in the modqueue view rather than the comments so you see the targeted harassment reports on "xyz is a meanie head" and just click "remove" because it already is on the edge at best for most subreddits. this is how context gets lost. so people would see "unfavorable treatment" (not that it didn't happen, gallowboob's company's domain was soft-banned on reddit yet his subreddits had automod rules set to approve them) when if more people were as trigger happy on the report button a similar thing would happen

the admin problems with this are much worse because the comments tend to be looked at in isolation so saying "i'm gonna kill you", in isolation, looks without context pretty bad, but might be part of a joke chain or song meme that reddit likes to do every so often. take into account the fact that admins get whiny sometimes if your AEO removals are too high. then take into account the AEO guy's Tarot card reading and whether Mercury is in retrograde and you get a lot of mods who are a bit trigger happy, esp when people've gotten banned for approving stuff the AEO removed for dumb reasons

this somewhat led to a bit of an inflated ego with regards to reddit but eventually from what i see he left... at least under that username anyway.

There have been incidents where users who reported certain spambots were themselves banned for "report abuse". It's speculated the operators of those spambots pay money to Reddit to not be banned.

I have a 16 year old account and received my first account warning, ever, for reporting a user's comment (to reddit, not the sub) a single time and the admins disagreeing that it violated the guidelines.

I have screenshots somewhere, but it basically said if I continued to abuse the report feature my account would be banned.

Reddit is a publicly traded company and I sincerely doubt the company is taking some organized racket money on the side. But there is some serious conduct issues with admins, and I won't speculate about their motivations.

Granted they are publicly traded and sure organized racket money dwarves for a policy traded company. However, if we step back from the making money side, there are real sovereign reasons for wanting to explicitly and implicitly allow for bot networks to operate.

First and foremost is the destruction of organization of people. If you’ll recall, Reddit used to be an oasis of the internet. Growing up, I was always in awe of the intelligent and impactful discussions that would occur organically (something Hacker News can’t even rival). Now, it’s slop.

There is real value there that Reddit is offering the government. I’m sure other foreign governments (the globalists) also benefit from a weaker people. So I’d imagine it’s the kind of racket that is so high-up, secretive, and decentralized that there is no real culpability and everyone is aware (except the people).

Neat rabbit hole. Reminds me of having to deal with email spam - it was a similar deal with rule-based filters, ML scores, domain bans,IP filtering, browse fingerprinting etc and mishmash of ever evolving scripts surviving org and personnel changes. Glad i dont deal with it anymore as the frontier seems to be 2 fronts now with human and agentic spam.

Based on the current status of my shadowbanned account (I suspect a competitor in our space retaliating), it looks like `banall` only flags posts from the last 6 years.

Of course, nobody can view my profile anymore anyway (I'm waiting on appeal), but on my account, only posts from the last 6 years have the "Sorry this post was removed by reddit filters" message.

My takeaway:

> My test account (5 years old!) got banned immediately, and all of its post history got wiped too. RIP

I want to know the real string of the event I ever want to delete my account and content. This would be much faster than using a browser script to manually delete.

A lot of what they are doing now is around AI comments and posts, I know this because in some of my subreddits I have various automod filters that align with the way AI writes, I see it all the time where my automod removes something and reddit removes it again after (some kind of race condition), as well as tons of these accounts getting site wide bans.

In case you didn't realise, a massive portion of content on reddit now is LLMs.

I'm not a massive fan of how reddit has played certain hands in the last 5 years or so, but I do hope they win the war on dead internet theory.

I suspect their IPO is a sign that they’re throwing in the towel existentially.

Can't you just append ".json" to the end of any Reddit link and read all sorts of these fields?

No, the API will not return the admin-only removal reason. The code path that causes this is in the post.

Damn, maybe I can finally find out why my 10+ year account was globally (and retroactively) shadowbanned, even though the appeal was allegedly granted.

In the past, those post removals didn't even exist in the moderation log, so perhaps a reason could give me a clue... On the other hand, I'm taking a kind of emotional damage just remembering.

Is it still shadowbanned? If you go to reddit.com/appeal what does it say?

There's basically only one result you can get from an appeal which is "we have reviewed your account and we will not be lifting the suspension at this time."

The appeals process exists to fill a checkbox that says there must be an appeals process, not to actually unban anyone.

I have seen multiple people have their shadowbans reversed after I recognized it had occurred and alerted them.

I appealed a warning I was given for quoting a Simpsons episode (promoting violence, Reddit-wide and not from a sub) and was my appeal was granted.

As I said, the appeal was granted, or at least that's the message I got in reply.

However everything remained in the nether-realm and the appeals page claimed that my account was normal, and that therefore couldn't be used

No response from any support ticket either. They still advertised me IPO opportunities though...

That's strange. I've never seen that set of circumstances. Pop me an email on my profile and I can try to put you in touch with someone who might be able to help.

what's going on with this post? It was posted days ago and now its back on the frontpage again?

My guess would be the second-chance pool (https://news.ycombinator.com/pool, explained by dang at https://news.ycombinator.com/item?id=26998308)

Exactly that, you can tell by hovering over the posted time "9 hours ago".. and see that it's really from June.

yeah but it was on the frontpage and quite popular last time! and the timestamp on the comments changed too, to be relative to the new "posted" timestamp.

"Anti-Evil Operations" is a pretty grandiose name for spam filtering. Also I liked the House of Leaves reference

ya it really is. it's the new name (where new is relative to like 10 years) for Trust & Safety.

"Anti-Evil Operations" is the name of their wrong think filtering, which is hilarious in a newspeak kind of way.

some more information perhaps?

banned_by true is more accurate to say "admin or automatic". in "admin mode," you can see these although not sure the UX for these nowadays now that it is spewing a gazillion lines of text into them).

Anti-Evil Operations removals (nee Trust & Safety) are generally human(-assisted) actions (although these actions can be applied en masse). there's some more information nowadays in the API which was really nice. it also helped because people stopped blaming "the mods" for removals when the spam filter slopped all over the place. this was also annoying because previously you had to previously guess from the API how it was removed even if you were a mod.

the 3 ways to remove a post/comment (i.e. in reply to: train_spam):

- remove not spam: removes it but doesn't train the spam filter, obvious

- spam: removes it and trains the spam filter, obvious

- confirm spam: only happens when you remove after removing for any reason, *does not* train the spam filter

- reinforce spam: trains the spam filter even if the spam filter already caught it. *does* train the spam filter. you can do this by doing `action: spam` in automod. not sure if there have been any more in the last few years

also you can tell the legacy of "removals", back in the day stories were "banned" instead of "removed" by moderators and administrators.

also also also... you can see a lot of the stuff from this article in the `approved_by` side of it as well. if you hover over a checkmark of someone who has been unshadowbanned, you'll see it says "approved by Reddit (shadowban removed)"

if an admin manually unspams someones stuff (say someone who got accidentally shadowbanned and got hit with an overzealous spam filter multiple times >.>), it'll say "approved by <username> (all)". there are some consequences to this. it approves stuff that has been "filtered" (as AutoMod filtering is a weird hack where it removes something but keeps in the modqueue).

> spammit

i believe this is the thing that is "pretty similar to a naive Bayesian classifier"[1][2] that reddit used. /u/Deimorz iirc was a reddit dev at the time and it was somewhat public info. i say somewhat because you kinda had to be both interested in the this and probably be around the metasphere

iirc from some other comments i pieced together there are also per-subreddit spam filters. in the olden days sometimes they'd get way out of whack and you could ask an admin to reset it for you... or something idk

> em

guessing em in this case btw refers to /u/hueypriest, who was reddit's GM at the time

> would’ve been catastrophic for Reddit’s spam issues

the thing that surprised me at the time was just how bad reddit's spam filtering is. i did a small little thing at the time where i'd just look at stuff following some basic spam filtering rules (like stuff you'd probably get out of an artisinal spamassassin ruleset) and even that deluge was amazing to see.

like the ML stuff is cool and all but seriously 90% of this could probably still be solved with some basic rules. the profile hiding stuff didn't help either but that was way after my time.

[1]: https://reddit.com/r/TheoryOfReddit/comments/10ko5h/comment/... (2012)

[2]: https://www.reddit.com/r/modnews/comments/6bj5de/state_of_sp...

I swear I read this article 2 or 3 days ago and the comments on that post were also same as this post. Am I missing something here?

You're missing the second-chance pool https://news.ycombinator.com/pool which allows certain posts to reappear as if they were new.

Once again expressing my opinion that this is the worst anti feature of the site. Threads are like commenting in the void because most people are not going to be looking for replies to comments they made days or weeks ago. I see the true datestamp of the comment I replied upon upthread was not 3 hours ago, but 3 days ago.

The fact that they change the timestamp is also very stupid (yes you can hover and still return the datestamp, but this is by definition a dark pattern). These posts should preserve the timestamp vs masking it and even should be flagged as [Second Chance] in the title imo.

That doesn't show for me unless I click the link.

How does it work?

dang selects a post to be second-chanced, and this resets the timestamp on it so it appears again.

wait what the heck yeah, this is the same post as from a few days ago, i guess the comment and post timestamps got glitched??

That'a thing HN's second chance pool does. If you mouseover the "3 hours ago" or whatever on a comment, it will tell you the actual post timestamp.

That is insanely bad borderline hostile UI.

Before we started doing it this way, the threads would fill up with far more off-topic comments ("why is this 3 day old post on the frontpage?" and so on). That was a much bigger problem. Relativizing the timestamps to reflect the re-up time was our attempt to address that. Overall, it has worked ok—that is, while it does still lead to offtopic confusion and complaints in the threads (boo) there is far less of it than there was before (yay).

I'm open to suggestions of how to do it better! But you also need to consider the cost of adding explicit details to the UI. If we did that every time something like this came up, HN would have become an unreadable mess a long time ago.

I guarantee you've thought more about this than I have, but the first impression I had of the "second chance" pool was that it would essentially be a repost of the top-level post and not the comment threads. I think part of the reason people bring it up is because they see the same post PLUS the same comments with new timestamps and feel disoriented.

Oh, that isn't how I thought the pool worked at all. Invisibly rewriting dates feels very dishonest/disorienting!

maybe just adding something like (revived) to the timestamp, or a [?] link that explains what happened to the date? If you want to avoid adding info to the ui it will inevitably create confusion

Just display the post's original timestamp and optionally a second-chance badge, instead of the timestamp it was second-chanced.

That's the status quo ante, which didn't work, plus a UI element which most people wouldn't understand. My guess is this would lead to even more offtopicness.

There have been almost zero visible changes to the HN codebase in over a decade, but the one thing I would love to see them add is a little flag on the heading of the post to say it is 2nd Chance Pool to avoid all these comments every time this happens and everyone is confused :)

My friend got shadowbanned for posting a youtube link, part of a interview with Sascha Riley (the one where the explains the thing with the tent peg):

https://www.youtube.com/watch?v=84PHEMLab6g&t=2807s

You get shadowbanned for almost anything these days. It's not worth trying to use that site any more - I wrote it off as a lost cause, a playground for bots to talk to each other thinking they're talking to humans.

The Russian bots on there trying to demoralize the UK are pretty hilarious too.

Reply on news.ycombinator.com