Algorithmic Music Generation With Recurrent Neural Networks [video]
m.youtube.com
m.youtube.com
With images the technique works because we like looking at the dense artifacts. Millions of dog heads essentially copy and pasted onto any appendage that looks like it should be a head. It looks like drugs and overwhelms in the same way.
If you just take white noise and throw it through a tuned filter bank (which is in essence and in final effect what they are doing here) then you just get crappy audio.
The more standard and successful use of NN in composition is the use it on pitch series and compositional forms. Like feeding it all of Beethoven and then getting it to generate similar compositions. That's been going on for decades. You can do it with that kind of data.
But the thing about pop and electronic music is that the easily machine observable elements are not very interesting. Listen to the 4/4 kick and snare pattern in the video. Its boring as hell. (Other tracks can be just a kick and snare and they are amazing and we celebrate them as classics and play them for 20 years. Machines will never understand why)
What's great and essential are things like spatial relationships between elements in the mix: how does the surge of the compressed synth/guitar cause the beat to tumble outward and stir you up ? after a series of peaks in a synth melody then the next time it pulls back creating a space that pulls at your heart strings. you create a negative space that the listener goes into. playing with listeners expectations based on what songs, conventions and tropes they already know and respond to.
We haven't found a way to enforce a good prior for temporal data like sound yet.
There's also the academic musical equivalent - music professors using stock faddy techniques like serialism or (currently) number and group theory.
It's not that this is an impossible problem. It's more that the set of people who can code machine learning algorithms and understand music theory and are creative enough to invent new algorithmic techniques and to create more-than-listenable music is incredibly small - double figures, if that.
So progress in non-trivial computer music has been incredibly slow. The DSP side has been far more successful, because DSP is - in most ways - a much simpler problem.
> or (currently) number and group theory.
That was Xenakis back in the 1950s !
Xenakis is the exception. He's really the foundation of academic computer music and his music is amazing and moving and his compositional concepts are still being hacked out by music programmers today.
Sometimes I wonder if linguistics has more to offer composition than algebra (speculation, as I know next to nothing about (non-CS) linguistics).
> It's not that this is an impossible problem. It's more that the set of people who can code machine learning algorithms and understand music theory and are creative enough to invent new algorithmic techniques and to create more-than-listenable music is incredibly small - double figures, if that.
Are you pursuing something like this? Or know anyone who is? This is one of my main interests (alongside better interfaces for composition and well, making music). I'm actually back at University for my second degree to study this sort of thing.
If you have a blog or anything I'd be interested to find out...
The irony is that musicians are actually striving for, but failing at, reaching the perfection level that computers have.
And so for computers to sound more human like they have algorithms that make them more "sloppy"
What composition algorithms lack is not the ability to compose like humans but a life that will give them angels and a story.
Then again a lot of music is really formulaic anyway and computers are used for most of it. There is nothing in a few years that will hinder some sort of computer star to be born. But it's probably never going to connect with us the same way another human can. Not for now at least.
Quote of the month for me. Bravo =]
If you think music is [list of numbers] that can be made more "human" with a bit of timing randomisation, then of course it's all perfectly straightforward.
In reality there's rather more happening.
>What composition algorithms lack is not the ability to compose like humans but a life that will give them angels and a story.
No, the music basically sucks as music. The number of people willing to listen to it voluntarily without being paid to - usually as students or academics - is vanishingly small.
The story part only becomes relevant after that problem is solved.
And while it's true that music is formulaic, it's also true that computer music hasn't yet worked out how to copy all the details of the formulas - never mind produce original and memorable new formulas from scratch.
The best formula copier is probably Cope's EMI, and that sounds exactly like what it is - a slightly confused cut-and-paste cliche machine, not a human composer with a point to make.
Music becomes meaningful in the listeners mind, and the things that make it meaningful is both that it's formulaic (structure) and whatever the performer instills in the listenter.
> on what our model believes the world looks like
Its not even doing that. Those images are just simple 2D fields of pixels with color data.
It has no idea about any worlds. The spatial cues are way off, but we understand art (impressionism through messy expressionism, glitch) so we forgive it. And because they chose images with puppy eyes and we like puppies. Take away the animal elements and everybody would say the images were boring crap.
That statement seems to be overly strong. Perhaps we are far from having a machine that can figure out high level aspects of a song on its own, but I don't think it's implausible that with some more guidance (for example by learning also loop arrangements and filters instead of only the waveforms) these neural networks can potentially create quite interesting music today (especially in the EDM/IMD genre). This development might be scary because it possibly replaces human creativity to a large extent, but you can't stop it by claiming that it's impossible or that it will always be of poor quality. People have said the same when synthesized music arrived, that it lacks human aspects etc., and now it's has a high cultural significance, even though it uses things like auto-tune and consists of super clean loops.
This is what I mean - its not about the loop and sample arrangements.
I've been doing exactly that since the 90s and I've made some great twisted computer generated stuff. Logical or predictable methods will always result in logical and predictable music. Its all about setting up code environments so that you can generate accidents and mutations and capture them. Its all about reacting and capturing it and putting it on wax (historically speaking).
But that doesn't mean that a computer can understand why or even recommend what is amazing. We humans don't even know why some things are so great. A major thrill is finding some sound that is so twisted (ill, stoopid, sick) and totally bypasses the rational mind, shuts your thinking down and you get a big smile and start jumping up and down acting like an idiot.
Then somebody else copies your track, then it becomes a style, then it becomes a cliche, then beatport is filled up with boring copie, then it becomes a sample set that people can buy, then somebody makes an app that can auto-generate that style and then they claim that computers are making music.
But they are just playing it back, just like tall the human copycats further upstream.
And the entire network of software, creators, audience and cultural is what we call music.
> This development might be scary because it possibly replaces human creativity to a large extent, but you can't stop it by claiming that it's impossible or that it will always be poor quality.
That's what I've spent a lot of my life engaged in. It often makes great music, but the machine cannot understand why. You'll need strong AI for that and it will need to have tensions, depressions, a body, chemical feelings, sex drive, longing and a strong psychological need to be lost in song. Then it could say "David, I think I've found a song you might like."
What is scary is the willingness of some people to accept dubious randomly-generated artifacts as "art" and as evidence of consciousness and intelligence. You should not trivialize actual human intelligence and creativity simply because you want to believe in strong AI.
GP post makes a very good points and (like usual, it seems) they get completely ignored in replies.
Setting up some 4/4 beat is easy. And you don't need an AI to throw is a randomly arpeggiated chord in there. It will sound okay-ish. Maybe it will even sound better than the stuff in this video. But that doesn't mean it is equivalent to proper music composition or that anyone would actually listen to that stuff outside of a technical demo.
BTW, if you want to see real procedurally generate music that is recorded by musicians for real listeners (as opposed to by CS grads for other CS grads) you might want to take a look at Karma: http://www.karma-lab.com .
> the music would actually give them the same satisfaction they receive from manmade music
You believe this because...? None of the generated tracks I've heard so far is anywhere near the kind of music people would listen to for the sake of the music itself.
Some people do listen to music to drawn out distracting conversations around them. Perfectly sensible thing to do. However, in those cases music can be replaced with white noise of nature's sounds. This does not demonstrate that music is equivalent to noise.
...
The problem with such AI demos is that they lack a particular purpose (i.e. success criteria), use existing material for training and do not go a step beyond that material. You can take a track and slow it down by a random fraction to get a different pitch and tempo. Viola. "New music." Procedurally generated. But that does not mean you've composed something. You've used a randomized algorithm hand-crated to produce a particular effect. It has no generative power.
Music doesn't need to be as coherent as a text (in fact some space for interpretation is often preferred), which also increases my confidence about this prediction.
humans don't really understand why either, if it didn't stop us, why should it stop a machine?
If they aren't doing it that way, I'd be interested in hearing how it's evolving the signal in that given direction - and also how that filter works (what libraries does it use?).
Sounds like it hit some sort of local maxima, so this system won't ever produce the original song, but something a percentage of the way toward it.
I'm a bit more interested in algorithmic composition, but this could be interesting if trying to blend genres. For a long time I've wanted to build a program that could produce essentially an infinite song morphing between genres with lots of tunable parameters.
We compare the performance of two different types of recurrent neural networks (RNNs) for the task of algorithmic music generation, with audio waveforms as input (as opposed to the standard MIDI). In particular, we focus on RNNs that have a sophisticated gating mechanism, namely, the Long Short-Term Memory (LSTM) network and the recently introduced Gated Recurrent Unit (GRU). Our results indicate that the generated outputs of the LSTM network were significantly more musically plausible than those of the GRU.