Whenever there's a discussion about copyright and AI-generated images, there will be people claiming that AI can't possibly reproduce images it has been trained on. Stable Diffusion and similar software supposedly only learns higher concepts and hence no one can claim that its outputs are derived works. The rationale I often see mentioned for this argument is that the model is very much smaller (in terms of bytes) than the set of images used to train it, so verbatim images can't possibly be stored in it.

Yet, there are plenty of examples where it will reproduce well-known works out of it. People have reported images with recognizable watermarks [1]. How do you reconcile this fact with the previous claim? Is there a special part of the model that stores Mickey Mouse, Batman and Getty Images watermark verbatim, but other less-well known artists' art gets reduced to mere concepts?

When you put in a prompt "magnificent owl" and get out an unrecognizable owl artwork out of the machine, I wonder. Is that really a novel artwork, or is unrecognizable merely because it was done by some random person uploading their art to an unknown DeviantArt account 5 years ago that was seen by 10 people in total since, just barely changed enough by the AI that a similarity search through billions of images in the training set won't find it?

[1] https://news.ycombinator.com/item?id=33044611