great distinction. can we use it to freely produce and share mp3/h264 copies of any media? because it is provably impossible to reconstruct original based on that information.
But keep in mind: So far there is no way to train the models while completely avoiding memorization and only including generalization. That would be a great way to avoid any copyright issues, but all attempts I have seen so far were fairly limited.
Because I really want to see difference between approximating input signal using smooth orthogonal basis functions, and approximating those using quantization with rate/distortion feedback as "violates copyright, compression algorithm", and then approximating input signal using perceptron, with backprop loss based off original media as "not compression algorithm, skip jail"