Hacker News Reader

Comment by Ajedi32 | Hacker News Reader

I find the idea of AI-based image compression rather fascinating. In theory it seems like a great fit for AI; "perceived quality" is exactly the sort of fuzzy metric AI tends to be good at optimizing for, and a compression algorithm possessing an understanding of high-level visual features like shapes, lighting, etc seems like it would have a lot of potential.

Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data; which could lead to some really bizarre looking compression artifacts when the system breaks down.

View on HN

> Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data

Or, you could have a compression system created specifically for photographic images (as opposed to a “general compressor” like LZ77), where these high-level features are part of the (de)compressor itself.

Now, lossy domain-specific compressors are pretty common; one good example is audio “voice codecs” for telecom, in which every snippet of audio is encoded as a phoneme-like symbol, such that noise comes out the other end as a gobbledygook that sounds a lot like words.

But much more interesting to me is a lossless domain-specific compressor, which I’m not aware of any examples of. (Please tell me if about some if you know them!) In such a compressor, the input would be “rebased” against the model, such that the output looks like an instruction stream ala “start with [this synthesized template with these parameters]; and then add point-modifications {A, B, C, ...} to get from there back to the original input.” So inputs that were more like something it “understood” would compress well (i.e. just be described by a template) while inputs that were less-well-understood wouldn’t get much from the procedural-generation phase, and would end up being specified entirely as point-corrections, and so not really compress at all.

It's pretty easy to build a lossless learned encoder. Encode the image with your model, then apply a lossless encoding to the error. You can also include a measure of the entropy of the error in your model's loss function.

The model should make the errors small, and including the entropy measure of the error in the loss function means that the model produces "cheaper to correct" errors.

A nice keyword on this front is "bits back argument".

That's a nice theory, but what if your ML encoder uses textures to encode perceptual equivalence efficiently that's actually got very high errors measured by pixel? It seems to me the best ML-driven encoders would work by inferring higher abstraction details from the image in ways which are hard to get back to lossless - that there's an intrinsic tension there.

Then choose a different encoder, or add another layer to shift/correct the texture.

A relevant NeurIPS 2019 paper for lossless compression (for e.g., medical imaging):

https://papers.nips.cc/paper/9383-integer-discrete-flows-and...

Yes, exactly! One of the winning entries at http://prize.hutter1.net/ (challenge to compress English Wikipedia) is/was a LSTM that predicted the next character then used arithmetic coding on the final bitstream (correct bit / incorrect bit).

Specifically, see https://arxiv.org/abs/1609.07959

FLAC for audio is one

That was my first thought too. To see how it leverages domain-specific properties, see:

https://xiph.org/flac/format.html#interchannel https://xiph.org/flac/format.html#prediction

What you're basically describing is compressive sensing. The basic idea is that if you can find a domain in which your data is maximally sparse, then you can transmit the same information with many fewer bits. Unless you're extremely lucky with your basis functions, it is not strictly speaking a lossless compression algorithm.

> But much more interesting to me is a lossless domain-specific compressor, which I’m not aware of any examples of. (Please tell me if about some if you know them!)

A bit academic, but if you check out this paper from 2015 https://arxiv.org/pdf/1502.04623.pdf on a neural network trained on binarised MNIST (28x28 pixel images), Table 2 quotes entropy figures of 116.815 after converting to bits, which would be 28*28/116.815=6.71x compression (if used as a compressor)

> Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data

I think that's the major risk for straightforward "AI" compression. You'll end up with some high level description of the scene and the decompressed image will then be an "artist's impression" which contains details that may not have been in the original image. You'll take a photo of a cheque, compress it, decompress it, and you'll get a picture of a cheque, but it might not be the same cheque.

In theory it is a great fit for ML. But not for the reasons you're suggesting.

The success of a compression scheme ultimately comes down to its ability to estimate p(x), for some sample x. And estimating p(x) is precisely the problem the field of ML addresses.

the idea of learning as compression is actually an old one. Take a look at https://en.wikipedia.org/wiki/Minimum_description_length

The tricky part is how to measure the perceived quality. With video there is VMAF but seems there has been no development on the image quality assessment.

Reply on news.ycombinator.com