Hacker News Reader

> WordPress with a cache can handle on the order of 1k requests per second on the cheapest VPS money can buy. An actual proper static site would likely 10x that.

Yes we know - I still had my site knocked out a couple of times by overeager bots. They can be really abusive.

View on HN

Wow, that's nuts. What sort of site was this? Mine don't get anything near this level of attention!

Just a personal blog, and I have to admit it had a misconfiguration that was my fault - nonexistent urls would redirect to the index. Still, dumb scraper bots would 1) not realize there was a redirect loop and kept trying every possible url, and 2) they were doing so aggressively enough that it brought the site down a couple of times. A well-behaved crawler would have slowed down and noticed the loop, never had issues with crawlers for years until the advent of ai.

Reply on news.ycombinator.com