Hacker News Reader

Comment by amelius | Hacker News Reader

MP3 is inherently a one dimensional codec, whereas JPEG is two dimensional. No wonder it performs much better.

JPEG transforms 2D images into 1D arrays using a "zigzag" ordering of 8x8 blocks. See this diagram on wikipedia:

https://en.wikipedia.org/wiki/JPEG#/media/File:JPEG_ZigZag.s...

It's a shame the author didn't do the same transformation, because it would de-correlate a lot of the error noise. You can see in the highest compression settings that the "MP3" image compression is smearing everything horizontally. If it used a zigzag transformation, it would be a more smeared both horizontally and vertically, but probably less visually bad.

Good point. Do you have any idea why they didn't opt for a Hilbert curve, like others suggested here?

The way the quantization matrix is set up is that most non-zero values end up in a corner, so the zig-zagging actually manages to be pretty good since the data is already suitably aligned.

I'm not the parent, but I'd probably surmise that images are usually encoded in blocks to allow the image blocks to be decoded in parallel.

If you're asking why they use the zigzag (within a block) instead of a Hilbert curve, IIRC (quite fuzzy on this, so take it w/ a gran of salt and verify) the reason is that it allows for better spatial encoding (imagine having a ripple in one corner and going out - that's essentially what you want to encode w/ your DCT). Using a Hilbert curve would preserve locality, but I don't think it would line up with the spatial distribution of frequencies in an image.

JPEG transforms 2D images into 1D arrays using a "zigzag" ordering of 8x8 blocks.

No. The zigzag ordering is applied to the frequency components, not to the image pixels.

What if you try to compress the frequency components, scanned in zig-zag, using MP3 (without the first FFT like layers if they exists, I guess)? - if that even makes any sense...

That doesn't really make sense. MP3 takes inputs in signal-space, not frequency-space. You could run:

1. Divide image into blocks (as in JPEG),

2. Perform two-dimensional FFTs (as in JPEG),

3. Scan frequency components in zig-zag order (as in JPEG),

4. Run all of the steps of MP3 compression aside from the initial "split audio into blocks" and "perform FFTs" stages.

That would pretty much just give you a less efficient version of JPEG; both JPEG and MP3 take advantage of knowing how much each frequency component "matters" (i.e., how precisely it's necessary to encode the value to avoid artifacts noticed by humans), so using the MP3 quantization logic on frequency amplitudes from images would result in wasting bits by encoding certain amplitudes more precisely than is useful.

You're right. I'm remembering a different codec — I think this is how H.264 orders DC coefficients for macroblocks. JPEG uses an actual 2D DCT, not a 1D DCT of a flattened block.

You could iterate a bit by leveraging 'joint stereo' mode [1], eg, by considering pairs of rows as left/right channels. And perhaps even further with more 'channels'.

[1] https://en.wikipedia.org/wiki/Joint_(audio_engineering)

I wonder how mp3 would perform against a Hilbert-type path taken through the image.

JPEG uses this path for its blocks, rather than anything fancy and Hilbert-ish: http://www.johnloomis.org/ece563/notes/compression/jpeg/tuto...

I think this alone would be an improvement for the mp3 encoder.

One thing to try is to map 2D image data to 1D using Hilbert curve or some other locality-preserving space-filling curve.

Not only that, but doesn't MP3 remove "sounds changes you can't hear" or whatever, whereas jpg removes "color changes you don't see"? There is a parallel there, but I would imagine they would be quite different since one is based on human auditory response vs visual response.

A psychovisual model.

Hmm, idea... suppose you ran tempo/beat detection on some music, and then took its 1D waveform and chopped it up and arranged it in a virtual 2D space, lining up similar sections (e.g. choruses would lie side by side). Could you then gainfully compress the result with a 2D compression algorithm?

No, because things aren't gonna live up that nice.

I had this idea a while ago and later discovered that the MOD file format beat me by a few decades :)

Also, today I stopped halfway through importing a FLAC of repetitive electronic music into Audacity, and I was surprised to discover that there was almost no repetition (it would play one bar then skip ahead over the identical parts)

Edit: Whoops, totally wrong!

Turns out I was loading a half-torrented file and forgot that torrents don't download in order!

Reply on news.ycombinator.com