put a honeypot link in your site that only robots will hit because it’s hidden. make sure it’s not in robots.txt or ban it if you can in robots.txt. setup a rule that any ip that hits that link will get a 1 day ban in your fail2ban or the like.
I have to make sure legit bots don't get hit, as a huge percent of our traffic which helps the project stay active is from google, etc.
https://github.com/pinballmap/pbm/blob/302ac638850711878ac61...
https://github.com/pinballmap/pbm/blob/302ac638850711878ac61...
but it only bans for 3 hours. if they don't respect the hidden link and robots.txt they get banned.
The it's the permanent iptables rule, but could be CF API call as well.