That would be on what the AI model generates, not what it is trained on.
The latter is where the contention is, and it's a valid argument. So much so that some companies are not using stolen information to build their models.
IBM for example indemnifies its models for its customers and has detailed information on where the sources came from to train them.
Arguably LLM companies could have made large-scale deals with libraries and got the exact same knowledge (much, much more slowly). I wonder if people would have the same issues then? My guess is probably. Goes back to the meme that if libraries were proposed today there's no way they would ever be allowed.
Your analogy is flawed because you are saying the information was free to begin with.
That analogy works for IBM granite models because the information they trained on is free to use.
The major LLMs did not do that.
It has been tried in court several ways already. Remember the lawsuit that forced Anthropic to use physical books? They tried to argue that the books couldn’t be trained on at all. It failed.