Hacker News Reader

Comment by tdeck | Hacker News Reader

For adversarial problems like this, a shadowban approach can sometimes be necessary. Perhaps people can still see their blogs but GoogleBot gets blocked from indexing them, or they only appear to someone with the spammer's cookies. That way it takes them longer to catch on and evade the model.

Of course, that means you'll need to at least spot check your bans because you can't rely on legit users escalating to you.

View on HN

Yep, shadow ban was in place as well.

The thing is, the people that weren't the ones posting the content. It appeared their computer was affected by some type of malicious file to be part of a bigger network. (botnet?)

I could see from the thousands of different IPs in different countries around the globe, that it could be affected personal computers.

Very few of them were computers from hosting companies, the rest were normal people computers.

I'm sure these machines were doing the job, someone else would have tests the result.

When we did the shadow banning it didn't made a dent in their effort.

The way they changed email, changed username, tried to be unique was completely prepared specifically for our platform (I would guess so)

Whenever we counter their attack, they would be silent for a while and then attack again. They would adjust.

Shadow ban is effective when the attacker themselves will not be aware it, in our case it was tricky to know who was the observer.

Did they map to ISP ASNs? Country geolocation doesn't say much anymore since there's so many VPN providers whose business is to buy a CIDR in every country and resell access.

Yes, almost all of them mapped to ISP ASNs.

Very few of them were from AWS, OVH and other hosting providers, very very few.

We ran each IP towards black lists ips, paid IP reputation checkers. Majority of the IPs were clean.

Back then we had IP reputation check, but it was a headache to maintain, so we disabled it later, however that time very very few them got stopped at IP reputation checks.

As someone that's used IP proxying services that provide millions of IPs for scraping purposes, that is a very mature industry, and they advertise (and I believe them) "millions" or IPS, even for what you might consider hard to supply ones, like mobile IPs, and they let you slice and dice them however you want? Datacenter IPs? Residential IPs? Mobile IPs?[1] What state or city would you like them in? Would you like the site you're hitting to not have been accessed by this IP (through proxying at least), and if so how many days? Do you want some mix of that? Make your own configurations and set them up as proxy endpoints, etc.

Fighting against abuse at the level of IP address attributes seems like a losing game to me. Honestly, the best I saw at this (3-5 years ago at least) for traffic was Distil networks, where they put a proxy device in front and examine your traffic and captcha or block based on that.

Since you have content being submitted, there's a lot more you can use to classify, such as how you used ML, so that's good. Part of me worries that this is all sort of reminiscent of infections and antibiotics though. The continual back-and-forth of you finding a block them finding a workaround feels kind of like you were training the spammers (even if you were training yourself at the same time). At some point maybe we'll find that most the forum spam is ML generated low information content posts that also happen to be astroturfing that is hard to distinguish from real people's opinions.

1: Fun fact, to my knowledge anonymous mobile IPs are provided by a bunch of apps opting into an SDK (like an advertising/metrics SDK) which while their app is open (at least I hope that's a requirement) registers itself to the proxying service so it can be handed out for use by paying proxy customers. Think about that next time you play your free "ad-supported" mobile game.

Yup they utilize "residential proxies" to hide their behavior which makes the situation worse because sometimes it will affect your legit users

I wasn't familiar with the term, so I just searched the phrase "residential proxies", and the space seems even more sketchy than the usual VPN peddlers who promise the world and more. Are they using malware-infected PCs, or what?

Some of them use the endpoints using their app. Which means a lot of people become the spammers just by joining the network. Of course it's mentioned in the middle of their T&C, but who reads that...

The most legit ones use their own users (to access the residential proxy you have to agree to become an endpoint yourself).

More shady but probably legal ones use JavaScript and Apps that typically hide the fact you are becoming an endpoint in the T&Vs.

Straight up illegal ones use compromised computers and routers like you'd think.

I’ve always been curious how the residential proxies work. Is someone going around paying people to run proxies in their home? Are these compromised devices being exploited? Are there a bunch of storage units someplace with cable service? The mind runs wild.

This seems like a lot of effort? What do you think was the motivation for something like this on your site?

The cynical answer is backlinks haha

We shadow ban abusive users on Fly.io and it works great. Everything seems to work right up until they try to connect to a deployed app.

It took me a while to realize that ramping up the frustration level is, itself, a helpful deterrent.

Are you doing this because of resource usage?

If resources are free then you could even actually deploy their app and either whitelist it for their own IP or only allow very few requests before taking it down.

This would be even more frustrating and could ruin whatever they plan to do with their abusive app in the first place. Let's say they deploy their malware/phishing page, test it a couple of times (possibly from a different IP) and it works. They then start spamming the malicious link and waste decent amounts of time/money/processing power, not realizing that the link was dead after the first 10 hits.

We're primarily trying to prevent fraudulent payments combined with expensive VMs. Throttling CPU to almost nothing on high risk accounts sounds delightfully irritating.

We also get the less resource intensive, but still harmful abusive apps that port scan the internet. Those are relatively easy to detect. We generally don't want to be a source of port scans so we shut them off pretty quickly.

I wonder if you could return bogus but plausible results to port scans? Either whitelist a set of safe ports such as HTTP(S?) so that if they try to "curl google.com" to confirm whether everything is OK they get a good response but silently drop everything else causing their scan to return negative on all ports.

How do you know the payment was fraudulent? Is there any possibility it can affect honest users?

Yeah, in the end you wind up with a small set of persistent adversaries who have been tweaking their abuse alongside your fixes, and a hopefully much higher wall for new abusers to scale.

If possible, it can help to hold back new systems and release a bunch of orthogonal anti abuse systems at once. Then the attackers need to find multiple tweaks instead of just evading one new system.

I wonder though, would it not be easier, to identify the watchers and shadowban those.

Just present bisected realities to spam watching groups and identify those who trigger a reaction.

I remember an old mailing list discussion on sourcehut, because sourcehut provides a build service that you can use for automation.

The decision from sircmpwn was, at the end, to charge money for the service. Charging money and KnowYourCustomer will kill most exploits dead.

In this sense, this is turning the frustration level to 11. You can use the service to a certain extent, without frustration, but if you want to get serious then you're going to have to jump through some hoops.

Dedicated people will still find a way through, but you've cut off 95% of the flow and killed the low-effort attempts. Now, you can focus on the serious shit.

> The decision from sircmpwn was, at the end, to charge money for the service.

Frankly, I'm shocked the other major free providers (GitHub/Lab) haven't done this by default. GitHub's current default is free for public branches, and a small fee for private.

I could see a flipped setup working: by default a very small fee (1-5 cents per x number of batches) that most companies wouldn't notice, and a path for FOSS projects to apply for credits.

I can't find the direct link, but I remember someone on HN pointing out that because CI tools are turing complete, GitHub actions is the cheapest serverless cloud product in the world right now -- you just need to figure out how to game the system.

I'm sure they've built very sophisticated filtering tools, but imagine someone slips through the cracks and gets a cryptominer working. Get that action registered in enough projects (by, say embedding it in an Actions library or generator tool) and that could be significant.

> I'm sure they've built very sophisticated filtering tools, but imagine someone slips through the cracks and gets a cryptominer working.

It's been done, multiple times. Here's a handful (96) of documented cases which are somewhat recent. [0][1]

It seems to be surprisingly easy to abuse the process, and GitHub are continually playing catch up.

[0] https://dev.to/thibaultduponchelle/the-github-action-mining-...

[1] https://www.bleepingcomputer.com/news/security/github-action...

Sounds like a micropayment system where there is much more significant first buy in amount to be on the micropayment system than the actual (tiny) cost of a micropayment to use a service would make it worth far less worth it to spam online services.

The risk of having their payment identifier/address banned from services before significant use makes it very risky for them to use such a thing even if tiny micropayments a worth it to the spammer.

It certainly could have other problems with people getting banned from such a system for things other than: spam and other use detrimental to the service provider. There is also the issue of how the initial buy in fee is distributed.

But a high buy in for a system that many online service providers use would very strongly discourage use detrimental to the services providers (I think; this is only for discussion, as this is posted by someone with little knowledge on this. Micropayment systems have been talked about a lot, but I don't remember high buy-in mentioned).

Edit: Forgot to mention that the idea of this is that service providers can offer their services at lower cost because the risk to them from a account/address with a high buy in is lower than from an account/address with no buy in.

Is there any possibility user is unaware she is "abusive"?

For example, I quickly searched the ToS for port scanning and didn't find it banned, yet you admit to intentionally frustrate such users.

I am not Fly.io customer at this time but I do port-scan my own services as a realtime security check.

As otherwise great service, you may open yourself for very angry user reviews if the tactic is "frustrate the user".

I'd think a serious attacker would easily detect a shadowban, but they'll be intensely frustrating for legit users. Seems like a poor tradeoff.

I’d presume that large scale spammers check their work in incognito via a different network. It’s their job.

check their work... It’s their job

The idea of running UAT for spammers/scammers/troll-farms is pretty depressing.

You would think, I’ve run into plenty of click bots that continue to run for years despite receiving noting but 4xx or 5xx results. You would think someone someplace would monitor that and spin the IP but no reaction at all.

Then there are others where its only an hour or so before rates get adjusted to our threshold and/or new IPs start emitting the same requests.

Reply on news.ycombinator.com