Ask HN: Is this idea for audio compression technically feasible? | Hacker News Reader

Ask HN: Is this idea for audio compression technically feasible?

8 pointspcf13 comments

I have a great idea for an audio compression algorithm, but does exist already? And if not, is it technically feasible?

My idea is that you take repeating identical sounds/frequencies (kick/snare/hi-hat/bass etc.) and encode them as the "same" as the first occurence in the song. That way you would save a lot of data/space, especially with electronic music where many of the sounds actually are 100% identical along the song.

But would this work in practice? Or would it be too complex to find out what is truly identical and what is just similar, but not similar enough to be marked as "same"?

I would love to hear what HN has to say about this. Thanks!

View on HN

As someone who listens to EDM (electronic dance music) almost exclusively (though, I'm no DJ or producer) the vast majority of those sounds come from some sort of midi type software or machine be it a drum machine, sound effects machines or software, or recordings of sounds that occur around us. EDM music also has a build up - the beat generally starts simple and gets more complex as the track progresses. Sometimes, thats called the drop, right? When a track builds up, they "drop the beat" and the best gets more complex, as found in Dubstep.

So, I think compression like that might work in software they use to write / develop a track; if the software can determine that 2 notes or beats are identical, it could code the file with less data in it, but I think of music to be similar to water. It's fluid, with each note / beat / voice effecting the notes / beats around it. Ever listen to a track with the voice removed? Or with the drums removed? It sounds very different then when the track untouched. Point being, I don't think this would work for a final mp3 or lossless compression.

(or I may be completely wrong)

You're kind of thinking of the "source code" for electronic music.

To simplify things: You start with a sequence of notes stored as MIDI, which are played through synthesizers and sampled instruments, which generate the sounds. Then you layer effects on top of that, such as reverb, filters, and compression.

Distributing this "source code" probably wouldn't make music files any smaller, since you would also need to distribute all of the sample packs and effects plugins. A single reverb or compression plugin will take up tens of megabytes. Acoustic instruments are usually sampled at very high quality. A grand piano library can be gigabytes of samples, which includes all kinds of variations for velocity, acoustic response, etc.

In addition to that, anything you record with a microphone can't be compressed in this way, such as vocals, or any acoustic instruments. So if you have vocals throughout the whole song, then the entire length of this audio track will have to compressed at the original bitrate anyway. So if you have a 3 minute song that uses lots of samples, but also has 3 minutes of vocals, then the vocals are your bottleneck. Whatever you do to split up your track into samples, your minimum filesize will be 3 minutes of compressed audio.

So as soon as you have anything that can't be reduced into repeating samples + filters, you won't be able to do better than MP3 or FLAC (for lossless compression).

If your song contains very little recorded audio, and is purely MIDI + synthesizers, then this technique could save a lot of space.

The analysing part sounds like something "The Infinite Jukebox" [0] must be doing to find suitable paths to jump between.

[0]http://labs.echonest.com/Uploader/index.html

I know little of the topic, but the end result sounds equivalent to methods that convert a music recording to a MIDI file. There is a list of such programs at http://wiki.audacityteam.org/wiki/Midi#Converting_from_audio... . That section does comment that this is a challenging research problem.

I should add that I'm a music producer myself and know very well how music is created.

What I'm wondering about if it's feasible having an algorithm that searches for repeated instances of any sound (don't limit yourself to thinking MOD or samples) across a file and then marks these as "same" in order to save space.

Any programmers who could tell me that?

Some people actually create such files by writing music with a tracker. http://en.wikipedia.org/wiki/Music_tracker

What you describe may be called a conversion to a tracker format.

Is this not the type of thing that every compression algorithm does?

Nope. MP3, for example, cuts off all frequencies outside the audible range and thus achieves large savings.

A 44.1Khz .wav file has no frequencies outside the audible range.

How well do you hear 1Hz? A 44.1KHz doesn't have a lot of frequencies outside of the audible range, but that's not really that relevant, as that's only a minor trim compared to the crazy things lossy compression algorithms do here. Their main focus is having the result sound almost like the original. The audible sound-image is modified extensively by them, with focus on simplifications that are hard to notice for a human.

AFAIK, every other audio compression encodes each slice of sound separately. My idea looks at the song as a whole, which I believe would be something new in the audio field.

Why don't you start by taking a waveform, rip it from a CD or something, and start looking at the time-domain signal to see how many of these repeats there are.

That's roughly what compression algorithms (zip, rar, arj, etc) do, identify repeated bits of information and store where there are ocurrences of these.

For lossy audio compression algorithms like mp3 and ogg, they use psychoacustic techniques, removing information the human ear can't hear, like very high or very low frequencies (that's why they're lossy). MP3 does something like slicing the signal in small fragments, applying some mathematical functions to them, like the Modified Discrete Cosine Transform (similar to Fourier Transform or FFT), then removing some components of it.

The technique you're describing would pretty much be equivalent of using samples and sequence them, which would be akin to reinventing the .mod format (.mod, .s3m, .xm etc)

To use samples though, you need the individual samples. To identify and extract the samples from the song would be much more difficult, unless maybe each instrument was in its own track or channel.

Reply on news.ycombinator.com