Hacker News Reader

The point of the post was that you need to add robots.txt (or similar) because it's a thing, and also tell us where they are.

> add a robots.txt

Which bots will then ignore.

I didn't expect I'd get to use this so soon. Obligatory canned advisory:

They still ignore it and scan everything even though you have told them not to scan certain paths.