One problem with Apple's approach here is that they were scraping the web for training data long before they published the details of their activities and told people how to exclude them using robots.txt
Robots.txt is already the understood mechanism for getting robots to avoid scraping a website.
Apple are saying you can opt out of their training data collection using robots.txt.
But... they collected their training data before they told people how to opt out.
I don't understand why me pointing that out as "eyebrow raising" is controversial here.
It might be nice if there were categories that well-behaved bots could follow, as noted above, but even then the problem exists for bots doing new things that don't fall into existing categories.
I think that's disingenuous of them.
Wanting to disallow LLM training (or optionally only that of closed-weight models), but encouraging search indexing or even LLM retrieval in response to user queries, seems popular enough.