Hacker News Reader

There's a difference between someone running a scraping tool occasionally and bots constantly and rapidly re-scraping the same site over and over again

View on HN

The author's website is responsible for storing its own data. AI services currently treat the entire web as their storage and cache layer.

This is an important context: the site is more likely to be targeted by scrapers because it is a curated collection of scraped information.

Do the bots care? Seemingly very little intelligence in many of them. Could be as simple as the site has more pages, so more traffic.

Loot first, ask questions later.

A curated collection of people who give away money. Was ever sweeter honey ever found in a pot?

Reply on news.ycombinator.com