Hacker News Reader

Comment by rco8786 | Hacker News Reader

I got DoSed by ClaudeBot (Anthropic) just last week. Hitting a website I manage 700,000 times in one month and tripping our bandwidth limit with our hosting provider. What a PITA to have to investigate that, figure it out, block the user agent, and work with hosting provider support to get the limit lifted as a courtesy.

Noticed that the ChatGPT bot was 2nd in traffic to this site, just not enough to cause trouble.

View on HN

At which level of DDos one can claim damages from them?

You can claim whatever you want, but actually litigation is expensive and it’s not at all a sure thing that “I made a publicly available resource and they used it too much” is going to win damages. Maybe? Maybe not?

So I could ddos anyone legally as long I have a some reason?

Sure. Courts tend to care about intent, but your are welcome to try to change that.

I've got claude bot blocked too. It regularly took sites offline and ignored robots.txt. Claude bot is an asshole.

robots.txt did not work?

bro, since when vc funded ai companies have the courtesy to respect robots.txt?

Of course it didn't work. At best, the dorks doing this think there's a gamechanging LLM application to justify the insane valuations right around the corner if they just scrape every backwater site they can find. At worst, they're doing it because it's paying good money. Either way, they don't care, they're just going to ignore robots.txt.

I have not monitored traffic in this way, but I imagine most AI companies would explicitly follow links listed in robots, even if not mentioned elsewhere on the site.

I’ve been doing web sites for thirty years, robots.txt is at best a request to polite user agents to respect the server’s desires. None of the malicious crawlers respect it. None of the AI crawlers respect it.

I’ve resorted to returning xml and zip bombs in canary pages. At best it slows them down until I block their network.

Reply on news.ycombinator.com