It wasn't copied - it was viewed, which is perfectly legal.
The AI does not have a copy of any of the the images it learned from. It merely learned styles and concepts of many images with associated text for those styles and concepts.
I'm wondering if you could erase an artist from a trained model by adding gradients in the opposite direction, erased from the sum of gradients as if it wasn't part of training.
Weights are usually randomly initialized, defining a function. Many images (which are randomly batched on each epoch) are passed through, and a loss function tallied. Backprop gets a gradient, and a training schedule tweaks the weight a little bit. No image has its "gradients" somehow added. The overall function has gradients, but they are not smooth, due to many things such as ReLU being non-smooth. These are not invertible in any sense.
Th networks have many, many irreversible steps in them that truncate, that max pool, that perform dropout during steps, many of these choices done randomly. There are gradient normalizations to clip or stretch to deal with vanishing or exploding gradients, and SOOO many lossy things done that are completely irreversible.
So none of this can possibly work. A network is not simply a sum of images in some format.
The model doesn't save the image itself, but eats the gradients. The question is how we see this process, does it entail a copyright penalty?
No, that would be incredibly slow. The loss is summed over items in a batch, then the gradient is computed once based on the loss, then backpropagation is done. It's done this way by default both in TensorFlow and in Pytorch.
It is possible to sum per entry (see here [1] in pytorch docs) but is is extremely rare. The majority of training loops (this one included) is the standard "do a batch" then call "loss.backward()" which computes only once per batch.
Look at the "backward()" call in pytorch docs [2]: This "Computes the gradient of current tensor w.r.t. graph leaves." If this were done per item and being summed, there would be no need for this in the training loop, and nearly every training loop I've seen has this.
Here's pytorch explaining all this in detail [3].
And in any case, since the functions in the pipeline contain many non-smooth terms, the information required to restore an image, even if only one were done, is not in the gradient, any more than the slope of a line tells you the x-intercept.
Another way to think of it: even for perfectly smooth functions, derivatives lose information that cannot be recovered.
[1] https://pytorch.org/functorch/stable/notebooks/per_sample_gr...
[2] https://pytorch.org/docs/stable/generated/torch.Tensor.backw...
[3] https://pytorch.org/tutorials/beginner/blitz/autograd_tutori...