Youtube Playlist: https://www.youtube.com/watch?v=_lcCJzfXl50&list=PLaa32nLgVv...
SoundCloud: https://soundcloud.com/songshtr/sets/machinecroon
Github: https://songshtr.github.io
The main challenge for me so far has been post-processing. Jukebox produces very "scratchy" / lofi outputs... and I've been DIY'ing with Audacity. Thinking of getting a track professionally mastered to see just how far the boundary is...
Yes I wrote the lyrics myself (other than a couple quotes from Eliot & Borges) - no GPT etc involved in the text. Even with the tunes, the seed music fed to Jukebox gives some control over what gets generated next... especially when you later splice various outputs together. Each ~2-3 minute song needs anywhere between 15-100 2-3 minute raw snippets to be generated from Jukebox before you can find enough sections to paste together. I suppose I would be considered a writer/producer while Jukebox is clearly the performer - an immensely talented, yet completely capricious one!
In the meantime, would point you to the Jukebox discord server run by Brocaloo as the best all-in resource: https://discord.com/channels/766622617393430559/769885769891...
check it out and hopefully is helpful!
The thing that's missing is the data. If we had midi transcriptions of 100k songs (abc notation could be fine too), we could probably get really interesting stuff, but most of what is available is lossy chord transcriptions and classical music (public domain). So if you want to automatically create something that sounds like mozart, you're in luck!
But this isn't really satisfying to me. For generative music, we're still largely stuck with encoding musical rules in code rather than feeding data to a transformer. To me, the former feels much less like AI than the latter. The data is all locked up behind an impossible quagmire of copyright.
But if I were a sheet music publishing company, I would be seriously considering the future of music creation with AI given my broad access to notated music & metadata (is this an original score, or a grade 1 simplification?). But again, music copyright is a pretty complex contraption.
So MIDI songs were being generated quite "early" on, even with basic text generators (LSTM models), where the "alphabet" was replaced by MIDI symbols. [1]
What Jukebox (and other models) did well was work with raw audio, rather than rely on MIDI or similar transcriptions. They break down audio into "blocks" using Fast Fourier Transforms, then each block is "tokenized" and then fed into transformers, similar to GPT-3 or other newer text generation models. This now allows the "subtle" musicalities to be discovered by models without relying on transcription. Wavenet (from Google) [2] I believe was the first one to do this... I tried myself for 6 months, but the processing power required for this is candidly only available for the Google/OpenAIs of the world.
>> the data is all locked up behind an impossible quagmire of copyright
So my understanding is that the training of Jukebox is similar to Copilot. It was trained on the entire English Spotify catalogue so all that has been digested by the model - under Fair Use. But again - this is probably far from "settled" legally.
[1] https://arxiv.org/abs/1612.07837
[2] https://www.deepmind.com/blog/wavenet-a-generative-model-for...
Whether it’ll be OpenAI or someone else is a tossup though. There’s not much commercial incentive to make AI music. So my money is on the hackers. Intellectual curiosity is still one of the most powerful forces of change.
The resources are there. You can get a couple hundred TPUs from google for free. So it’s a matter of talent, not permission.
How?
edit: There's also this on ambient endless generative music that I think was an HN submission a couple of years ago https://generative.fm/
https://www.youtube.com/playlist?list=PLv7BOfa4CxsHAMHQj0ScP...
Launching soon
Join the discord https://discord.gg/a5ttYuG
I used it to remix one of my favorite songs ever by Biz Markie just before he passed last year. It ended up being the last remix made of his music before his passing, and I sent it to his manager just before, so I'd like to believe he heard it.
It still has a ways to go (as audio quality can be spotty and incomplete), but these online services can often separate more than just vocals now, they can cut individual instruments out of music, and even create pretty good instrumentals. I have been able to remove uncleared vocals from fully mixed tunes that I've made so they can be released as well. I never thought it would have been possible 20 years ago when I started music. As for AI generated music, I think it will be a travesty to de-value or remove humans from the music/art making process entirely, it will always likely be something derivative of human work in essence anyway, but I don't think it will ever match the depth and soul of human-generated music to people who truly know and love music, some things just can't be emulated.
Here's the remix video - https://www.youtube.com/watch?v=xL14JH5f-qM
https://apps.apple.com/us/app/e%C5%8Dn-by-jean-michel-jarre/...
Article from BBC about the app: https://www.bbc.com/news/entertainment-arts-50335897
XJ is human created content fed into a "player" that fuses the composition process with the storage and distribution. It's an algorithmic medium. The music is never "done" and what you hear in the app is playing live in real time from the software we run in the cloud.
What do you think??
Free on iOS and Android.
That said, I don't have anything to show yet... Only got a landing page at https://tunesage.com at the moment... but here's an ancient prototype writing very generic melodies just to prove I'm trying... https://www.youtube.com/watch?v=8cwEaiwEZfU
Regardless of my personal eventual success or failure in the endeavour, the field does seem to be ripe for innovation! Nothing out there at the moment (that I've seen at least) seems to satisfy my current desires as a composer.
From my perspective as a trained, practicing musician MuseNet produces quite credible results: https://openai.com/blog/musenet/. But we're not yet there; AI generated music is still recognizable as such. Another, earlier project with very good results was https://www.microsoft.com/en-us/research/publication/automat...; unfortunately the website is no longer active.
Some nice results, here's the "About" page that describes their process:
https://ooo.ghostbows.ooo/about/
----
Edit: Here's where Robin Sloan mentions Jukebox:
I've toyed around with it a bit. It's impressive for sure, but I am not sure I think of it as anything other than a curiosity.
https://gnossiennes.mousereeve.com/
It generates an endless version of the famous minimalist piano piece.
It's really just a matter of time before they figure out how to make the music a bit more "cohesive."
Right now it's kinda like a stream of consciousness, vs. being verse, chorus, verse, chorus.
My prediction: in under 5 years, most of us will have a favorite AI band.
Also available on iOS
Technically not ‘AI’ though, more based on music theory ‘rules’.
Of course there exists rule-based AI. Whatever works to partially replace a human professional.
> The music from Generative.fm is composed by a human—not AI
edit: just had a look. it appears to now be the magenta project which is referenced elsewhere in this thread.