Hacker News Reader

For adversarial problems like this, a shadowban approach can sometimes be necessary. Perhaps people can still see their blogs but GoogleBot gets blocked from indexing them, or they only appear to someone with the spammer's cookies. That way it takes them longer to catch on and evade the model.

Of course, that means you'll need to at least spot check your bans because you can't rely on legit users escalating to you.

We shadow ban abusive users on Fly.io and it works great. Everything seems to work right up until they try to connect to a deployed app.

It took me a while to realize that ramping up the frustration level is, itself, a helpful deterrent.

Are you doing this because of resource usage?

If resources are free then you could even actually deploy their app and either whitelist it for their own IP or only allow very few requests before taking it down.

This would be even more frustrating and could ruin whatever they plan to do with their abusive app in the first place. Let's say they deploy their malware/phishing page, test it a couple of times (possibly from a different IP) and it works. They then start spamming the malicious link and waste decent amounts of time/money/processing power, not realizing that the link was dead after the first 10 hits.

We're primarily trying to prevent fraudulent payments combined with expensive VMs. Throttling CPU to almost nothing on high risk accounts sounds delightfully irritating.

We also get the less resource intensive, but still harmful abusive apps that port scan the internet. Those are relatively easy to detect. We generally don't want to be a source of port scans so we shut them off pretty quickly.

I wonder if you could return bogus but plausible results to port scans? Either whitelist a set of safe ports such as HTTP(S?) so that if they try to "curl google.com" to confirm whether everything is OK they get a good response but silently drop everything else causing their scan to return negative on all ports.

How do you know the payment was fraudulent? Is there any possibility it can affect honest users?

Yeah, in the end you wind up with a small set of persistent adversaries who have been tweaking their abuse alongside your fixes, and a hopefully much higher wall for new abusers to scale.

If possible, it can help to hold back new systems and release a bunch of orthogonal anti abuse systems at once. Then the attackers need to find multiple tweaks instead of just evading one new system.

I wonder though, would it not be easier, to identify the watchers and shadowban those.

Just present bisected realities to spam watching groups and identify those who trigger a reaction.

I remember an old mailing list discussion on sourcehut, because sourcehut provides a build service that you can use for automation.

The decision from sircmpwn was, at the end, to charge money for the service. Charging money and KnowYourCustomer will kill most exploits dead.

In this sense, this is turning the frustration level to 11. You can use the service to a certain extent, without frustration, but if you want to get serious then you're going to have to jump through some hoops.

Dedicated people will still find a way through, but you've cut off 95% of the flow and killed the low-effort attempts. Now, you can focus on the serious shit.

> The decision from sircmpwn was, at the end, to charge money for the service.

Frankly, I'm shocked the other major free providers (GitHub/Lab) haven't done this by default. GitHub's current default is free for public branches, and a small fee for private.

I could see a flipped setup working: by default a very small fee (1-5 cents per x number of batches) that most companies wouldn't notice, and a path for FOSS projects to apply for credits.

I can't find the direct link, but I remember someone on HN pointing out that because CI tools are turing complete, GitHub actions is the cheapest serverless cloud product in the world right now -- you just need to figure out how to game the system.

I'm sure they've built very sophisticated filtering tools, but imagine someone slips through the cracks and gets a cryptominer working. Get that action registered in enough projects (by, say embedding it in an Actions library or generator tool) and that could be significant.

> I'm sure they've built very sophisticated filtering tools, but imagine someone slips through the cracks and gets a cryptominer working.

It's been done, multiple times. Here's a handful (96) of documented cases which are somewhat recent. [0][1]

It seems to be surprisingly easy to abuse the process, and GitHub are continually playing catch up.

[0] https://dev.to/thibaultduponchelle/the-github-action-mining-...

[1] https://www.bleepingcomputer.com/news/security/github-action...

Sounds like a micropayment system where there is much more significant first buy in amount to be on the micropayment system than the actual (tiny) cost of a micropayment to use a service would make it worth far less worth it to spam online services.

The risk of having their payment identifier/address banned from services before significant use makes it very risky for them to use such a thing even if tiny micropayments a worth it to the spammer.

It certainly could have other problems with people getting banned from such a system for things other than: spam and other use detrimental to the service provider. There is also the issue of how the initial buy in fee is distributed.

But a high buy in for a system that many online service providers use would very strongly discourage use detrimental to the services providers (I think; this is only for discussion, as this is posted by someone with little knowledge on this. Micropayment systems have been talked about a lot, but I don't remember high buy-in mentioned).

Edit: Forgot to mention that the idea of this is that service providers can offer their services at lower cost because the risk to them from a account/address with a high buy in is lower than from an account/address with no buy in.

Is there any possibility user is unaware she is "abusive"?

For example, I quickly searched the ToS for port scanning and didn't find it banned, yet you admit to intentionally frustrate such users.

I am not Fly.io customer at this time but I do port-scan my own services as a realtime security check.

As otherwise great service, you may open yourself for very angry user reviews if the tactic is "frustrate the user".

Yep, shadow ban was in place as well.

The thing is, the people that weren't the ones posting the content. It appeared their computer was affected by some type of malicious file to be part of a bigger network. (botnet?)

I could see from the thousands of different IPs in different countries around the globe, that it could be affected personal computers.

Very few of them were computers from hosting companies, the rest were normal people computers.

I'm sure these machines were doing the job, someone else would have tests the result.

When we did the shadow banning it didn't made a dent in their effort.

The way they changed email, changed username, tried to be unique was completely prepared specifically for our platform (I would guess so)

Whenever we counter their attack, they would be silent for a while and then attack again. They would adjust.

Shadow ban is effective when the attacker themselves will not be aware it, in our case it was tricky to know who was the observer.

Did they map to ISP ASNs? Country geolocation doesn't say much anymore since there's so many VPN providers whose business is to buy a CIDR in every country and resell access.

Yes, almost all of them mapped to ISP ASNs.

Very few of them were from AWS, OVH and other hosting providers, very very few.

We ran each IP towards black lists ips, paid IP reputation checkers. Majority of the IPs were clean.

Back then we had IP reputation check, but it was a headache to maintain, so we disabled it later, however that time very very few them got stopped at IP reputation checks.

As someone that's used IP proxying services that provide millions of IPs for scraping purposes, that is a very mature industry, and they advertise (and I believe them) "millions" or IPS, even for what you might consider hard to supply ones, like mobile IPs, and they let you slice and dice them however you want? Datacenter IPs? Residential IPs? Mobile IPs?[1] What state or city would you like them in? Would you like the site you're hitting to not have been accessed by this IP (through proxying at least), and if so how many days? Do you want some mix of that? Make your own configurations and set them up as proxy endpoints, etc.

Fighting against abuse at the level of IP address attributes seems like a losing game to me. Honestly, the best I saw at this (3-5 years ago at least) for traffic was Distil networks, where they put a proxy device in front and examine your traffic and captcha or block based on that.

Since you have content being submitted, there's a lot more you can use to classify, such as how you used ML, so that's good. Part of me worries that this is all sort of reminiscent of infections and antibiotics though. The continual back-and-forth of you finding a block them finding a workaround feels kind of like you were training the spammers (even if you were training yourself at the same time). At some point maybe we'll find that most the forum spam is ML generated low information content posts that also happen to be astroturfing that is hard to distinguish from real people's opinions.

1: Fun fact, to my knowledge anonymous mobile IPs are provided by a bunch of apps opting into an SDK (like an advertising/metrics SDK) which while their app is open (at least I hope that's a requirement) registers itself to the proxying service so it can be handed out for use by paying proxy customers. Think about that next time you play your free "ad-supported" mobile game.

Yup they utilize "residential proxies" to hide their behavior which makes the situation worse because sometimes it will affect your legit users

I wasn't familiar with the term, so I just searched the phrase "residential proxies", and the space seems even more sketchy than the usual VPN peddlers who promise the world and more. Are they using malware-infected PCs, or what?

Some of them use the endpoints using their app. Which means a lot of people become the spammers just by joining the network. Of course it's mentioned in the middle of their T&C, but who reads that...

The most legit ones use their own users (to access the residential proxy you have to agree to become an endpoint yourself).

More shady but probably legal ones use JavaScript and Apps that typically hide the fact you are becoming an endpoint in the T&Vs.

Straight up illegal ones use compromised computers and routers like you'd think.

I’ve always been curious how the residential proxies work. Is someone going around paying people to run proxies in their home? Are these compromised devices being exploited? Are there a bunch of storage units someplace with cable service? The mind runs wild.

This seems like a lot of effort? What do you think was the motivation for something like this on your site?

The cynical answer is backlinks haha

I’d presume that large scale spammers check their work in incognito via a different network. It’s their job.

check their work... It’s their job

The idea of running UAT for spammers/scammers/troll-farms is pretty depressing.

You would think, I’ve run into plenty of click bots that continue to run for years despite receiving noting but 4xx or 5xx results. You would think someone someplace would monitor that and spin the IP but no reaction at all.

Then there are others where its only an hour or so before rates get adjusted to our threshold and/or new IPs start emitting the same requests.

I'd think a serious attacker would easily detect a shadowban, but they'll be intensely frustrating for legit users. Seems like a poor tradeoff.

When we had this problem, we added an input element positioned off-screen and ignored submissions that populated it. Cleaned it all up.

This is usually called a "honey pot" if some people want to search tools for implementing it.

Only works if it's generic spam not targeted to your site in particular. Otherwise they will just adjust the script to not fill it.

This story needs to be turned into a writeup and submitted to HN on its own.

If your eyes can "normalize" a unusual symbols to a common one to make an English word then so can a lookup table. I feel like this isn't a case where you'd reach first for a neural net.

Those are known as homoglyphs. This issue has been studied and best practices are documented. There's no complete solution but it can be mitigated.

https://en.wikipedia.org/wiki/Homoglyph

All kinds of possibilities there.

Unicode Character “;” (U+037E) is a beaut.

https://news.ycombinator.com/item?id=28621681

Yeah, then someone has to create or find that whole table and make.

The initial problem wasn't those symbols but the content itself, the symbols and special characters came into the problem later.

Later on as mentioned in my original comment, that they would use positive content from other blog posts that were published/passed the moderation to mix up their bad content.

Probably could use a different method, but at that time needed something quick and fast and it worked and still works with very little tweaking.

Although we don't have massive amount of threats or abusers anymore to exactly know the effect, but again, so far it works.

That time, they would coming several thousands per minute, IP blocking, range blocking, USER AGENT, captcha or anything such didn't work on them.

The good news is that the Unicode consortium has a report on this issue, and the tables already exist for normalization and mapping of confusables to their ASCII lookalikes: https://www.unicode.org/reports/tr39/

Oh, that's nice.

I guess I can use that next time time to work on the data cleaning for that model.

Thanks.

As I commented above:

I built a Python library for finding strings obfuscated this way. Was critical when moderating our telegram channel before an ICO. https://github.com/wanderingstan/Confusables E.g. "𝓗℮𝐥1೦" would match "Hello"

In fact, the Unicode consortium provides a report and extensive list of "confusable" symbols, which you could use alongside Unicode normalization tables to map adversarial back into more ASCII-equivalent text before running it through anti-spam mechanisms that are interested in the content of the message.

https://www.unicode.org/reports/tr39/

I built a Python library for finding strings obfuscated this way. Was critical when moderating our telegram channel before an ICO.

https://github.com/wanderingstan/Confusables

E.g. "𝓗℮𝐥1೦" would match "Hello"

I should have had better eyes while searching, could find this and saved some hours.

I only learned about it myself after spending too long building my own half-baked version. I think it's in pretty opaque language that makes it hard to find even if you know what you want.

Maybe somebody else on here will see it and learn about it before they need it, and at least you still have a new tool to reach for in the future.

You should post this to Show HN. Also you have a typo on your README ("characgters")

Thanks, I've fixed the typo! It was such a simple project, hardly seems worthy of a "Show HN".

I've seen crazier things get to #1

Test for the library: would it catch that that typo still refers to characters?

The one built-in to Python will get you most of the way there:

    >>> import unicodedata
    >>> unicodedata.normalize('NFKD', '𝓗℮𝐥1೦𝗵𝗲𝗹𝗹𝗼')
    'H℮l1೦hello'

Obviously it isn't going to remap leetspeak characters like 1 -> l but it covers a lot of cases.

Obviously you're saying it doesn't cover everything, but a big thing it's not going to catch beyond leetspeak-type situations is the kinds of thing you (used to) see in internationalized domain spoofing: legitimate non-Latin-script letters that just look the same or nearly the same.

NFKC/NFKD will handle "this is another form of the Latin letter A" type stuff but not "Cyrillic A looks like Latin A."

Thank you for making this!

What I don’t understand is who sees spam that looks like an old school ransom note and thinks ‘seems legit’, and then actually buys something?

There should be a 'bad symbols' list you can block like you do expletives. There's surely zero need to support that kind of thing in comments.

One term used to describe this is homoglyph, as in homoglyph attacks for phishing.

Back in 2015 I did some work just using simple bitmap rendering plus OCR to find text that looked like a small selection of known words. It was actually reasonably effective.

If you can identify text written with mixed glyphs just ban it outright. Normal users don't use text like this, the pure binary presence of such "homomorphic" text at all is probably a better signal for spam than whatever your neural net when running it after normalization.

> Normal users don't use text like this

They kinda do. Check out the shrug "emoji", table flip, and so forth. Then there's the meme of adding text above and below by abusing Unicode's "super" and "sub" modifications.

You could block it to only ever represent ASCII, but then you've knocked out the ability to expand internationally.

> Normal users don't use text like this

I think that depends on the users. People copying and pasting bits of text that was in English or another common language— think documentation, code, news articles, tweets, etc.— with a different character set could be problematic.

Also, 𝒮ℴ𝓂ℯ 𝒜𝓅𝓅𝓈 marketed as "𝔽𝕠𝕟𝕥𝕤 𝕗𝕠𝕣 𝕤𝕠𝕔𝕒𝕝 𝕞𝕖𝕕𝕚𝕒" would be ℭ𝔞𝔲𝔤𝔥𝔱 𝔲𝔭 𝔦𝔫 𝔱𝔥𝔦𝔰. (math symbols) A user base with young people getting bounced or shadow banned for trying to express themselves or distinguish themselves from their peers would be like ಠ_ಠ (Kannada letter ttha)

I think targeting the language they're using is a better bet.

¯\_(ツ)_/¯ (Hirigana letter tsu)

I can especially echo the "social media fonts" trend. They're quite popular on certain Discord guilds at least.

Minor nitpick, but ツ is the katakana tsu.

Oh, fact. Not a Japanese speaker. (or reader)

Bizarrely, I'm seeing recruiters using this on LinkedIn.

huh. For any specific purpose? Does it seem like they avoiding paying for recruiter accounts or something by evading algorithms designed to detect their activity, or is it just for the heck of it?

It's following some business wank advice to stand out from the crowd like printing your resume on A3 card stock.

Ah. Sounds like something one of those pickup artists would say if they blindly pivoted into marketing.

> pasting bits of text that was in English or another common language

If they use many (maybe three? four? or more) character sets in the same post, or different character sets in any single word, then that'd be highly suspicious?

Whilst still letting people copy paste from another language

Special case needed for the shoulder shrug with an Hirigana letter tsu I mean katakana tsu

I've noticed much more usage of alternative Unicode ranges for numbers/letters in email subjects lately to make marketing messages stand out, too (in addition to emoji of course), though I wouldn't necessarily mind banning that...

> Normal users don't use text like this

Sounds like you live in a filter bubble.

(╯°□°)╯︵ ┻━┻

(ﾉ◕ヮ◕)ﾉ*:･ﾟ

I know right? There are so many times when I've wanted to use something like box drawing unicode characters (cp437) to explain a complicated concept on hacker news, but alas I couldn't, due widespread computer fraud and abuse. How are we going to build a more inclusive internet that serves the interests all ALL people around the world, regardless of native language, if the bad guys are forcing administrators to ban unicode? (╯°□°)╯︵ ̲┻̲━̲┻

You can hardcode a rule for this specific bypass. Or you just retrain the neural net, and it learns that presence of these symbols = bad very quickly, and you spent less time writing and testing a custom solution.

> Found out, they have learned the trick and now using symbols from different languages to make it look like English.

I wonder if you can train a ML model using text as images. For example, taken as strings, "porn" and "p0rn" are not very similar, but visually they are.

That could detect bad text in actual images too

Not to mention “pom” in certain fixed width TTY fonts.

Are you familiar with the Xerox copying flaw? Not exactly the same problem and unlikely to apply here but still a fun tangent:

https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...

Do you have a mechanism for appealing the automated process?

what are they google? wait google doesn't even have that

My comment was targeted at companies trying to take Google's crappy approach at this problem.

Not sure if I understand correctly, but if you mean how re-training and deployment would be.

Nothing fancy tbh.

For the first several deployment while taking care of edge case and debugging, all manually on my own laptop and shot into cluster as a docker image.

Later, when starting to classify more content on the platform itself:

- Webhook will trigger the CI to train the model with new ham and spam content. - A new docker image was created and deployed to container.

The above would fail if success rate in validation/testing would be below 95%.

Still when classifying bad content, the whole process happens automated.

Special thanks to Gitlab CI, shell scripts, Python and Docker Swarm.

I think they meant (and I am interested in hearing about) appealing a "block" decision that was made by your automation.

If I'm a real human and trying to post a "good" post, but the model classifies it as bad and automatically blocks it, how do I appeal that decision? Can I? Or is my post totally blocked with no recourse?

Oh got it.

Thanks for clarification.

When a post gets published, it will be send to machine learning image via REST.

If bad, the post will be kept as Draft.

A new record gets created in another database table to keep track them, the accuracy rate was recorded as well.

This was made to make sure no irreversible action was done on the good content.

Blogs with more than 1 year of history would not go through moderation but no action was being taken, just recording the accuracy for future reference.

Later, someone from our team (me usually) would check them by eye and pull trigger on them, they would go into make the training better.

If something would pass the moderation but it was indeed spam, would go into another iteration.

We had to do this for over a month, through the time, the success was around 99%, no blogs would be wiped by machine classification from our database unless confirmed by someone.

That time the whole model was trained for that specific content. Later it get into other type of spams. Which we trained different models.

Overall, the the machine actions were logged, content/users/blogs would get labeled and bad marks on them.

They would be displayed in a report page, until someone make the final decision, through the whole time, the user would be shadow banned (shadow banning didn't help though) and their content would not be published.

Thanks for the detailed response! And nice to hear how much you've managed to keep humans involved in the process. I used to work on a content review automation system for a big company, so it's always fun to hear about how others handle similar cases.

And there's a lot of overlap between how that system acted and what you're describing. It makes we wonder if there's space for a company that offers this sort of model training + content tagging + review tooling capability as a service, or if there's too many variation on what "good" and "bad" input is to make it generalizable.

I think he's talking about false positives.

I'm interested in why these people were doing this. Were they hoping to get non-tech-savvy people that were searching for computer help? I guess that's a good audience of unwitting users to attempt to hack, but was the goal to get them to submit to one of the remote tech support scams? Were they embedding malware into your blogging platform, or getting ad revenue out of this somehow?

> Were they hoping to get non-tech-savvy people that were searching for computer help?

Yes.

They would create this posts and get quickly on search results (The platform is pretty good for making SEO optimization out of the box) and they would write good quality posts as well.

They would also share this posts on some other websites, especially social media accounts.

We don't have Google analytics or such to see where exactly they would come from. I noticed huge traffic to such pages by looking at the logs.

Our nginx log parser was alerting us about sudden spike on certain blogs and pre-defined list of words we have.

That's when we noticed something is going on.

Didn't take more than couple of hours (while working on the model) that we receive email from data center people about hosting phishing content, again didn't take much longer we received emails from some of those companies as well.

> Were they embedding malware into your blogging platform, or getting ad revenue out of this somehow?

No. On the blogging platform, we have everything bleached out, nothing would go in without passing through sanitizers.

They would simply had people convinced to call those US numbers.

I actually called one of those numbers and yeah, it was one of those customer supports some other part of the planet earth and definitely not from the company he was pretending to be and very quickly asked me to install team viewer on my machine. I really wanted to let them access it via the windows on my virtual box and have some fun with them, but well, someone had to fix the moderation issue :D

Why didn't you ban the user when you found out the scam?

We did.

We banned, shadow banned, deleted, recorded and many other things to them to make sure they're not reaching their goal.

The problem was, it wasn't a single user to deal with. thousands and thousands with no stop.

Flooding the platform like zombie apocalypse.

Jesus... what a nightmare.

Were these all using your free plans?

Oh yes.

Never ending.

We still keep the free plans even through there are abusers, but that won't be a reason to retire it. So many people using for legitimate reasons and keeping their personal writing there.

It's unfortunate, but well, it happens.

Why do you think they were using your platform to do this?

Is it because these scammers were non technical?

Or maybe the anonymity provided by your platform?

2 theories

1. Highly technical, because the flow was scripted to work with our website. Bypassing captcha, email verification by using many different domains, email accounts and also highly distributed via many ips.

2. Non technical. Where they paid some ppl to do it manually, which doesn't seems to be due to the way they walk through many steps like piece of cake.

However, our platform was/is a target due to several reasons:

1. Easy to register and start blogging. 2. Free plan with no hard limit. 3. Quick rankings due to SEO implementation out of the box. 4. Absence of any moderation before such attack.

And probably some other possible reasons that made their job easier and us a better target.

If I was this sort of scammer, I’d have the text being posted and progression of how hard it is to figure out pretty well ironed down. Then you just try it on lots of platforms to find the ones that don’t figure out how to solve the issue. So my guess would be they tried because they might as well, and then they hit the limit of how much effort they had put into their own generation systems and moved on to the next mark.

If you don't mind me asking, what sentence embeddings model (bert/roberta/etc) did you have the best luck with for your classifier? I like the quick retrain that can be done with an approach like this, though I have found that if you throw too many different SPAM profiles at a classifier it starts to degrade, and you might have to build multiple and ensemble them. The embedding backend can help a lot with that.

Tried bert but didn't get the proper result, probably wasn't working with it properly.

Here's the old source I have on my computer that did the training

https://gist.github.com/Alir3z4/6b26353928633f7db59f40f71c8f...

This was doing the early work and later changed more to fit other cases.

Pretty basic stuff.

How do you detect the ground truth for training the model? Do you manually label it?

Yes, simple classification. Nothing fancy.

Basically, pulled the database into CSV file and anything that was published before the bad content was classified as HAM.

We had content that were OK, so marked as HAM and then our new bad content all marked as SPAM.

When deployed to production for some hours HAM content got wrongly marked and model got trained on them as well which made so many confusion but the problem was taken care of once the model got properly tuned and safer to let it be automated.

Hmm I wonder if it picked up timestamps as its initial filter.

What was the motivation of those attackers?

I'm curious why you'd roll your own at all when there are moderation services available. Did you just have a use case that didn't match anything on the market?

Would you be able to share information about the tools you used, they could be very useful for other blogs and platforms it seems!

Basically https://gist.github.com/Alir3z4/6b26353928633f7db59f40f71c8f...

but don't think it will be useful for everyone, each usage and requirements is different and needs different solutions.

Thank you! You'd be surprised how useful a starting point can be, thanks for sharing.

What software/libraries have you used for your machine learning moderation system?

tensorflow and scikit-learn to train and build the model.

On the front FastAPI (behind uvicorn) to accept calls via REST API.

Deployed via docker.

To be honest, tensorflow and scikit-learn may not be the right fit for everything.

Every situation needs different approach and different solution.

Worth nothing, the most time consuming part was dealing with data itself and not model training or machine learning.

In couple of hours you'd notice you're starting at charts and tuning parameters.

"Worth nothing, the most time consuming part was dealing with data itself and not model training or machine learning."

Become a data scientist they said. Yay, artificial intelligence...

Hahaha. Yes, I felt that every second while dealing with it.

Did you try rate limiting, shadow banning, ip banning etc?

I wonder if it also ends up blocking legit content?

How did the lousy content affect your legit users?

Every blog is pretty isolated from other users.

It's not that the content will be popped up to everyone when someone posted something.

There's a Feed page where you can read what others you follow have published.

There's a Explore page where latest content without any filter or categorization would be visible. This is where such content would appear, but only blogs older than 7 days would appear there (we have removed that delay in recent versions).

Basically no one noticed them.

Although we disclosed the issue we were dealing with some of the old users of platform when they complained about their posts not getting published. That was the first issue in first 15 minutes of the machine learning model classifying wrongly due to being fed mixed content (where bad content was mixed with good content from those exact blogs by spammers.)

Other than several bloggers reporting their posts wouldn't go through as expected, no one else got effected and I hope no people were lured with those scammer while their content was published on our platform.

That sounds like a killer saas startup

and then your filter has a few false positives and you get cancelled as people claim you're anti <fill-in-the-blank>

Tale of humans man...

I would recommend you a different approach, such as using metadata like their location etc.

It was a coordinated attack from many different locations and IPs.

The IP blocking was in place on application level and later into Cloudflare blacklist. Still they would flood in with different IPs and browsers.

Still i would consider working on such metadata. I have seen many interesting results based on this.