Hacker News Reader

In this day and age, only good bots, the ones you actually want, obey robots.txt. You obtain no benefit by having a restrictive one.

That is true, but if you have Anubis you presumably do intend to `Disallow` all bots.

Not necessarily. You might want to rate limit bots instead of disallowing them.