I think more often it’s about translation. From image to classification ex. with no hope of mapping back.
The classifier still has to learn a model, which is a lossy, compressed representation of the world. Classifiers can be very lossy.
The outputs from Google's Deepdream are the kind of things that a classifier would "uncompress" to.
Yes, but not solely. They are also generalization.
If you let your eyes roll to the back of your head and let the spice take over generalization and compression are two sides of the same coin.
There are plenty of compression methods that have zero generalization. There are also methods of generalization that do not compress at all. It seems to me that they are orthogonal concerns, even if they may make sense to use together.