If a site allows crawling by at least one public-Internet spider, is there any legal protection it has against other crawlers who choose to ignore robots.txt and crawl anyway? Because I feel like that's exactly what Google would do here, as long as it wasn't literally illegal for them to do so.