ParentFull threadcatboybotnet·Seconding. robots.txt is just a way of "asking nicely." If somebody wants to scrape, they're spoofing their UA and ignoring it. Can't do anything other than monitor the logs and ban IPs one by one.View on HN