Artists look at images (copyright or not) while working on their own images all the time. Many of Shakespeare's works are re-writes of stories that were common in his era? How is this any different?
If the outputs were sufficiently different from the most similar training examples would it be exempt from copyright liabilities? On the other extreme, a single copyrighted image in the training set could spoil the whole model, it would be impossible to make sure the training set is 100% clean.
This is a great kind of question to ask, and can point us to the right areas of non-deep-learning copyright to think about – but not in a way that lends itself to a quick settled answer in a comment section, I suspect. https://law.marquette.edu/facultyblog/tag/fairey-v-ap/
> On the other extreme, a single copyrighted image in the training set could spoil the whole model, it would be impossible to make sure the training set is 100% clean.
Where possessing stolen goods is illegal, it takes quite a lot of effort to determine that an e.g. antiquities dealer has met the reasonable standard of inquiry for 100% of their inventory – but that doesn't mean that isn't the legal requirement. That's an area where we know the legal standard, and this is much newer ground, so it very well may be that tech that depends on datasets too big to take responsibility for may not end up being a great foundation for anything with legal liability attached.
Digital technology enables such massive economies of scale, doing away with almost any kind of friction you'd encounter when doing something in the real world, that they are qualitatively different from their analog, offline counterparts. Quantity has a quality of its own, as the saying goes.
I'm undecided on the morals of this, so don't understand it as an argument against doing it per se, but "I'm doing it perhaps a couple of hundred times daily if I really try hard, so it's clearly okay to do it billions of times a second" isn't a good argument for it.
If you believe in copyright then you don't believe in ethics^1 ;).
[1] Assuming you've actually thought about copyright from first principles—I think many people are too busy to have given it enough thought.