You're kicking the can down the road. Obviously, an AI is not a person, that's in the premise of the question. What makes an AI so different from a person that warrants differential treatment in this case? It's not that there aren't any good answers to this question, but yours is not much of an answer at all.

It's probably okay to learn facts from copyrighted material even if you're an AI - you can't reproduce the text of a novel but you can learn the meaning of a word from context in it. Similarly you can learn to draw a hand from looking at a ton of stock photos with hands, as long as you produce original hands.

AI will need some favorable legal precedents to avoid getting banned though, or else they'll have to only train off CC0 Flickr/Wikipedia scraping.

I also think it's a more obvious problem that it can reproduce copyrighted characters by eg prompting for "Homer Simpson".