what actually are the consequences of ignoring robots.txt (apart from DDOS)? have any of these cases ended up in court at all?
https://www.bbc.co.uk/news/articles/cy7ndgylzzmo
So an ai company can just be naughty till asked to stop, and then exclud that one company that has the financial resources to go legal.