People have sued over similar stuff in the past, like Google training their language model on copyrighted materials. The legal consensus is that a model trained on a large volume of copyrighted works does not infringe copyright, even if it occasionally vomits up chunks of the input model.

That said, if you trained a model specifically designed to violate copyright I suspect the courts would have a different opinion. They consider intent just as much as (if not more than) what you actually did.