Hacker News Reader

Use a user-agent containing a URL to find out who and what you are, and honor my robots.txt.

Having a panel for webmasters along with that would be fine.

Great suggestion... thanks for this one. We're putting this on our list

Please tell me that the robots.txt suggestion is something that you're already doing and the user agent part is whats going on the list.

You could try doing IP address rotation by rotation EC2 instances or some other cloud services.

I wrote a library for that.

That has nothing to do with what's being discussed on this thread.