Hacker News Reader

And by linking to your site from other websites, by not requiring authenticated human-only login, and by not blocking traffic from all referrers, you agree to allow my bot to access you site at zero cost.

View on HN

I have clearly communicated the constraints of use on my site using robots.txt, which is the established standard for communicating things like these. You are expected to fetch it and figure it out. It is not necessary to build walls and lock doors when a no trespassing sign is clearly visible.

Seriously, though, making robots.txt have the weight of law would be a massive improvement for the ecosystem in general.

and what happens when the perpetrator does not reside where that law can reach?

Reply on news.ycombinator.com