Hacker News Reader

Comment by twfarland | Hacker News Reader

Mindblowing... could this technique also be applied to music?

It should be, at least on audio level it would work. It would make a nice VST :)

Something similar does exist: http://labs.echonest.com/Uploader/index.html

It detects places in songs where the sound is similar, and can jump between these places in the track to make a seamless unending song.

I doubt it, because music is 1-dimensional and for 1-dimensional arrays WFC is just a Markov chain.

There are three-dimensional views of music, e.g. time-frequency-amplitude. See https://en.wikipedia.org/wiki/Spectrogram

Spectrogram is 2D (plot of amplitude given time and frequency).

Its interesting to think about it for a spectrogram because "similarity" is different in each dimension (freq vs. time). Frequency is also perceived logarithmically, so you would probably want to convert to e.g. Mel scale before applying this algorithm (a 2000-2100Hz change is much subtler than a 200-300Hz change).

Isn't that 3 dimensions (amplitude, time, and frequency)? The plot of course fills 2 spatial dimensions and uses color to represent the 3rd dimension. But I don't know very much about this.

I don't have a mathematically rigorous understanding of it but the number of dimensions is basically the number of freely varying inputs to the corresponding functional representation. In a 2d image, x position and y position are mapped to a color, e.g. I(x,y) = C. In a spectrogram, freq and time are mapped to a color (amplitude) e.g. S(f,t) = A. In neither case can you just pick an arbitrary color or amplitude and in general produce a singular x/y or f/t from that.

Another multi-dimensional abstraction for music that's at a bit higher level is a musical lattice. The number of dimensions is limited by the largest prime you accept in the ratios you use. (Typical western music approximated by 12-tone equal temperament uses primes of 2, 3, and 5. Powers of 2 are often ignored because notes an octave apart are perceived to be sort of equivalent for harmonic purposes.)

https://en.wikipedia.org/wiki/Lattice_(music)

How is music 1-dimensional?

A microphone records amplitude of sound waves over time, i.e. the air pressure -- that's the dimension.

You can also view digital images as 1-dimensional arrays of bits. (This is roughly how fax machines work.) That doesn't mean they can't also be 2-dimensional images, or representations of 3-dimensional images, or indeed an encoding of a 3D scene directly.

Similarly, you can unpack a linear sequence of sound samples into a two-dimensional plot of frequency and amplitude with a fourier transform.

Or you can represent the music as instructions to performers or synthesizers (ie notation) and you've got as many dimensions as you want.

Music is not sound, it's made of sounds. The fact that it gets mixed down to a single waveform when you consume it, either in the studio or when it hits your ear, isnt particularly relevant to how its made. I suppose the same is true of images though.

Markov chains applied to midi have been able to make "locally similar" stuff since forever. I wonder how this algorithm could be applied to higher-order aspects of music notation.

> Or you can represent the music as instructions to performers or synthesizers (ie notation) and you've got as many dimensions as you want.

These are just multiple signals in a single dimension (time).

1 physical dimension. Mathematically, each signal is a dimension.

We still call images "two-dimensional" when they're colored. There is a difference between continuous dimensions like space and time, and discrete dimensions like color channels in an image, or like instrument "tracks" of a song. The latter can have correlations, but they'll be sparse associations, rather than structural formulaic ones.

> Similarly, you can unpack a linear sequence of sound samples into a two-dimensional plot of frequency and amplitude with a fourier transform.

Fourier transform by itself is still 1D (amplitude vs. frequency); to get to 2 dimensions you can plot it over time i.e. a spectrogram.

You're describing audio, not music. Music as data isn't quantified in the same way as audio (see the various forms of musical notation that exist), and in fact music displayed in a sequencer or tracker looks pretty similar to the 2D bitmaps in the article.

Perhaps possible to slice the music wave data into frames, perform FFT to get a spectrogram and use that as tiles?

Reply on news.ycombinator.com