Hacker News Reader

Got a good link to something on github that does this?

I have to make sure legit bots don't get hit, as a huge percent of our traffic which helps the project stay active is from google, etc.

here's one example using rack attack on a rails app:

but it only bans for 3 hours. if they don't respect the hidden link and robots.txt they get banned.

I did it manually and got fail2ban to read the access log anyway.

The it's the permanent iptables rule, but could be CF API call as well.