We probably need a technical rather than legal solution for this. The problem is that generative algorithms are susceptible to accidental memorization so you can't guarantee that the output will be transformative. For example, play with
https://talktotransformer.com to see how many well known pieces of text it can spew out verbatim. It is very prone to derailing into a harry potter fan fiction regardless of prompt.
Other than copyright there are privacy considerations too. For example Gmail's Smart Compose is trained on users' private messages so you don't want it to memorize "private" details (such as credit card numbers): https://arxiv.org/abs/1802.08232
Is it possible to solve this by adversarially checking if the output is "original" enough or not? Or is that intractable, given how much resource our society already pours into making the same classification in court?