Hacker News Reader

Comment by kardos | Hacker News Reader

The obvious question here is: what does JPEG encoded music sound like?

Horrible. I actually tried applying image compression to sounds over a decade ago (was still in school, did that for Jugend forscht). I used a Fourier transform to generate image sections of a specific window of samples and used that. In hindsight, perhaps just treating the waveform as image data may have been better. The idea behind what I did was to see whether a spectrogram can be compressed well, considering that there's lots of repetition and blank space usually.

In any case, JPEG murders pretty much everything in thag representation.

It kind of reads like you have discredited frequency domain compression algorithms. FYI JPEG used a block-based DCT which is very similar to a Fourier transform.

That was not my intent. I just noted that my approach came from visually inspecting spectrograms and noting that they might be compressed quite well by image compression algorithms. Only to notice in the end that lossy image compression completely butchers the signal with noise if doing that. Had I just used the original waveform as raw image data that wouldn't have been as much of a problem. Compression would still have changed the samples, but overall probably not as badly.

Heck, I've been 17 when I did that. I certainly didn't really know much about the theory behind or why it almost certainly would fail. But I guess by now I wouldn't even attempt such craziness, so in a way it's probably a good thing to not know things too well, sometimes.

The problem is that we are much more sensitive to audio distortion than visual, particularly dynamic (e.g.video). Pretty much all lossy audio compression does a bad job without a psycho-acoustic model, whereas image compression works pretty well on simple metrics. It doesn't help that psycho-visual modelling is less well understood (than audio).

I'm more curious about a different idea. If you hook up electrodes to a tongue and then start activating its sensors according to the amount of light being received, people quickly develop the ability to see via their tongue.

So what happens when you encode visual information into sound in such a way as to have a sound-to-environment signal and you just keep that active and running real time. We already know that this mechanism exists in nature and can even be developed in humans, since echo-location is a thing.

Edit: It occurs to me that actually doing sensory experiments like this might really mess with someones head... maybe not a good idea to play around with it casually.

Edit 2: If we were going to swap out one of our senses for another, which should we get rid of and which should we gain?

Actually there are around (and have been for a few years aready) some glasses that scan the image you'd see and translate that into a sound that you need a bit training to understand as an image[0]. Some people have gotten used to them, while others have gone the way of the bat and learned to echolocate[1].

[0] http://gizmodo.com/these-synthesia-glasses-help-blind-people...

[1] https://en.wikipedia.org/wiki/Human_echolocation

Here's an Android app that converts images from the phone's camera to sounds as a tool for the blind:

https://play.google.com/store/apps/details?id=vOICe.vOICe&hl...

Your comment reminded me the TED talk "How a blind astronomer found a way to hear the stars"[1].

[1] https://www.ted.com/talks/wanda_diaz_merced_how_a_blind_astr...

It's been done and it works. I believe they just map the image to a spectrogram and convert it to audio: https://www.seeingwithsound.com/

Lots of noise. https://milek7.pl/.stuff/jpegcompress.flac (24bit PCM as 10000px wide RGB image compressed using JPEG quality 85, ~50% of uncompressed size)

Similar results - converting to a square image at 8bit then lowering JPEG quality just gets noisy in an uninteresting way:

    sox mo.wav -e unsigned -b 8 -c 1 -r 48k mo.raw
    bytes=`stat -f %z mo.raw`
    width=`echo sqrt\($bytes\) | bc`
    square_bytes=`echo $width \* $width | bc`
    dd if=mo.raw of=mo_square.raw bs=$square_bytes count=1
    gm convert -depth 8 -size ${width}x${width} gray:mo_square.raw -quality 50 mo_square.jpg
    gm convert mo_square.jpg gray:mo_square_jpg.raw
    sox -e unsigned -b 8 -c 1 -r 48k -t raw mo_square_jpg.raw mo_jpg.wav

Sounds like a weak radio signal! I wonder if we could pass it through a deringing filter to remove some of that noise, eg, like these guys have done: http://www.cs.rpi.edu/~oztanb/Papers/spie07paper.pdf

Not that bad, actually.

Just tried converting the sound to 8 bit, compressing it as a monochrome image, then back.

You can't really hear much difference at higher quality settings, and as you turn it down it sound like there is more and more digital-ish noise. I had to turn the compression way up before the voice became almost illegible, but you can still hear that it's someone talking.

I'd be curious what you consider garbage audio, as compared to 'not that bad.'

It's garbage when you need to switch to CW to continue the conversation. :)

Seriously, when it's unpleasant to listen to. As i said in the other reply, with music the quality is pretty bad on almost any setting, voice just happen to be an exception.

Tried again with music...

Even of high quality, there is a background sheesh kind of noise, similar to what you get if you play a 3D printed record.

On higher compression it's turning into something rather crumbly, with a recognizable tune but horrible sound quality.

So, yeah. Not quite usable.

> similar to what you get if you play a 3D printed record

I guess I'm behind the times, not relating to that reference...

Are you trying to fund a gif music player ?

Reply on news.ycombinator.com