Hacker News Reader

Comment by 1vuio0pswjnm7 | Hacker News Reader

"Cloudflare classifies bots by behavior, and a single bot can exhibit more than one behavior."

Is that really true

CF classifies anyone not using a popular browser with Javascript enabled as a "bot"

CF fingerprints www users

As an example, look at CF's Permissions-Policy HTTP response header on a site with CF "bot protection", i.e., the "checking your browser" CAPTCHA nonsense (challenges.cloudflare.com). Then look at IA's Permissions-Policy response header. One CDN is advertiser-focused, the other is user-focused

IA = Internet Archive

View on HN

As far as I can tell, after months of fighting being DDoSed by Anthropic and OpenAI across 50+ sites - Cloudflare also allows what it considers "good bots" through all of your bot blocking rules, with no option to turn this off unless you pay them money.

The "good bots" also just happen to be from companies that pay cloudflare a lot of money, I imagine.

Goodness Tokens

How did you dertmine it was those two companies? Also did you disallow them in robots.txt?

No, I definitely spent months fighting off bots across multiple hosts and never set up a robots.txt file anywhere nor did I look at the analytics dashboard on cloudflare.

I'm asking sincerely, how do you get to the conclusion that it's those two?

To that point, Cloudflare quite literally tells you the source of the bot traffic on your sites that use it as a WAF! And what percentage they are allowing through, despite you setting rules specifically to block them by name.

It’s baffling.

CF classifies anyone not using a popular browser with Javascript enabled as a "bot"

This is my biggest complaint about CF. They are implicitly supporting user-agent discrimination in favour of Big Browser, instead of discriminating on actual behaviour.

...and of course there are already companies running tons of VMs with "officially sanctioned" browser + OS stacks, that can get past all these "protections", for a fee.

"AI bots" is the newest boogeyman they came up with to take away freedom.

11,9,10

Reply on news.ycombinator.com