Hacker News Reader

I don't think cookies will do much against a large-scale automated attack, but everything else in this list is solid:

Hide posts until legitimacy of the poster has been verified. Allow them to post and respond, but don't show it to anyone yet, except to the moderators. If they're posting something sensible, unhide them. If it's spam, shadow-ban them. Don't let the user know. Let them guess why nobody is responding to the spam.

For that reason, it may also be a good idea to post an announcement that nobody should respond to this spam. Tat way the spammer won't know if he's being ignored manually or auto-hidden. Let him waste time and frustration on that.

Only use this against people who are this malicious. For regular hot-headed people who accidentally break forum rules but do want to meaningfully contribute to the community, always remain open and honest. Give people the opportunity to learn. Only people who are determined not to learn and to remain purely destructive, do you use the shadow ban.

Of course once they catch on, they'll probably start making new accounts with some legitimate posts, and once people start responding, they go back into spamming mode. This is tricky. Ideally, it'd be nice if you had a system that could automatically detect that sort of spam. If someone suddenly starts posting ascii art, bigimages, all caps, or anything like that, or goes on a rapid posting spree, automatically put them back on probation with hidden posts requiring approval, so you can check this change in posting behaviour.

We already tag/drop all Tor and VPN traffic. Cookies don't make much sense because this looks like browser automation, not someone just swapping VPNs by hand repeatedly.

For IP bans, they are now using illegitimately acquired or fraudulent IP space (the guy is not intelligent enough for this, he's almost certainly just buying proxies with in-game gold or some BS - but there is a criminal element here), including what might involve significant hijacking of AT&T, CenturyLink, Level3, and Windstream network resources.

(if you work at one of those places and are clueful, I would be very interested in asking about this)

Try the basics first, i.e. hiding of the posts from new accounts, IP and cookie blacklisting. See if it works.

Then move on to more advanced stuff, e.g. browser fingerprinting and behavioral pattern matching. If it's automated, there'll be a pattern. But the basic stuff can go a very long way.

As a bonus, you can put all new accounts into a single group, so that they would see each other posts, but without making these posts visible to the approved accounts.

If you're seeing connections from random residential IPs, they're probably using a reverse-proxy service like Luminati or 911.re. IP blocklists won't catch these. These proxies originate from (basically) compromised computers -- people who install "free" browser extensions and the like: https://www.trendmicro.com/vinfo/hk-en/security/news/cybercr...

With a troll this persistent (and willing to spend money on it), your best bet is definitely shadow bans and moderation queues.

It isn't just random residential IPs, but what appears to be BGP hijacking of/onto residential networks on top of that.

What's the basis for assuming BGP hijacking? That would be very sophisticated attack to just troll some random website.

The skid is not the one doing it. They are buying from semi-professional "proxy sellers" that do it and then sell you some form of authenticated squid proxy that further makes the request.

Can't he fake the source IP address by sending raw packets? Are they sanitized in any way?

That wouldn't work for HTTP(S) or anything else that works over TCP since the reply would go towards the fake source IP address, thus the attacker couldn't even get past the 3-way TCP handshake.

Can you elaborate?

If the troll is using residential proxies, you might try abuse@ the handful of services that offer such things. There aren't that many. I don't know if they actually take abuse seriously, but it can't hurt.

Luminati has an abuse form: https://luminati.io/report-abuse

Use js fingerprint library. Ban on the fingerprints. Black list the whole provider if it came from a hosting provider. You’ll end up blacklisting real users so you’ll need to ability to whitelist. WebRTC used to be able to pierce proxies to get the real IP via STUN, could try that. You could also check ttl on tcp protocol to see if it’s going through a proxy or client is lying on the user agent header.

There’s more stuff on the client you can do to prevent attackers from just hitting your APIs and forcing them to run a full client with having client solve sort a problem that it needs to provide to the API. You can detect headless browsers as well in JavaScript with a few open source bot detector JavaScript libraries.

By the way how do you detect VPN traffic? For tor we just pick up the list of exit nodes but we've had trouble identifying VPN without using a 3rd party API.

I know a site just banning a IP opening 443/tcp. It bans web server and possibly ssl-vpn server, but causes false positives.

we are a stealth startup founded by former Paypal and Coinbase engineers. We have a device intelligence product that can detect accesses from proxies/VPNs without using any IP list. We can also detect the True OS that someone is using - useful to detect emulators and script kiddies. Happy to chat if of interest: info AT sardine.ai

You can pick up a lot of VPNs by checking the ASN. If it's a hosting company, chances are it's a VPN or a bot.

Doesn't work for the VPNs that get you on residential IPs though.

maxmind has anonymous IP DB. it costs money

Shadow banning is for technically illiterate trolls not spammers. I would not advice automatic shadow banning as it might be psychologically destructive for false positives in a community. Technically illiterates will never understand that they are shadow banned and keep posting in vain and think they are ignored.

I would advice sending a registration token as a clear picture to the user email (really short one, like four letters). Then the spammer need to do pattern recognition and if he can't program he will not bother.

what's the problem if your posts don't show up to the public at first?

Also, with them using Selenium, I doubt that they can't put together some rudimentary pattern generation...

I have no problem with requiring manual validation of eg. the first post or what ever it is the shadow banning I don't like.

I adds complexity for the users and it is hard to know for user what is happening since it is a "secret" that someones messages doesn't show up. When the moderation process breaks down or there is a bug the user wont know how to appeal etc. It's the Google way of user interaction.

Just add a "This message will not be showed to others before your account has been approved" or whatever and it is not a shadow ban anymore.

The spam program might be shipped with Selenium. If the spammer can fix a trivial OCR text then OP knows that the spammer is a programmer not a "script kiddie".

Imagine a picture with four rows of letters and a rectangle around four letters. The rectangle moves around between registrations. Surely that would require a handwritten OCR to solve if the spam app ships with some boilerplate OCR? And be trivial to solve for humans with bad eyesight.

What if an actual user's blind?

Maybe include a url to request manuall approval and make the text white so only screen readers notice it?

I've seen a site like that, a Captcha + a manual form. I signed up through the manual one a couple years ago. I'm still waiting to hear from them.

Ye well if the moderation doesn't work properly any manual review wont work including whitelisting users first post.

The amount of legally blind users should be low enough that there can be a ad hoc queue of some sort. I guess any spammer can spam down that queue too though?

Maybe there could be a simple sound captcha for blind people with links to some audio files with letter in them to listen to in order. If the script kiddie figures out how to match byte streams add white noise to make it an DSP problem.

> Allow new accounts, but hide messages from them until their posts are verified manually

But they're registering hundreds of thousands of accounts. And, I assume, many of those accounts are creating posts. Moderating that many posts doesn't seem like a solution.

If it's a targeted attack, they'll just make one clean account to check if they're shadowbanned. The rest of the suggestions are great.

It's for the better if they discover that there's a shadow ban mechanism is in place. This acts as a deterrent.

Circumventing shadow bans is fairly annoying to handle correctly; you have to get a KNOWN clean account (which if it gets shadow banned, becomes useless without notice) and verify other posts. You have to wait some time after your post to avoid racing issues and there is some other tricky timing issues that means that this will heavily slow down any post (I mean, you already have to now check if your post went through, that's atleast 1 additional connection).

This can also be easily spotted and combatted by using a half-shadow-ban for accounts that are seen repeatedly browsing threads that just had a shadow banned post made; the user will not see random shadow banned posts, making detection even more difficult as you have to now cross reference multiple accounts.

You can make it arbitrarily hard to circumvent shadow banning.