DrumBot: Real-Time ML Drummer | Hacker News Reader

DrumBot: Real-Time ML Drummer

magenta.tensorflow.org

102 pointsmariuz19 comments

Some of my thoughts on this, as a fellow researcher in the space...

1) What selected the particular drum samples to use, and what methods were employed?

2) Can we assume that only Western approaches to time and rhythmic syncopation will be applied by this "real-time ML drummer"?

3) What about gestural performances, and non-standard approaches to timing? Can this detect and reconcile those kinds of use cases?

4) What does it consider to be "musical" input? Can non-musical input be used? If so, is the accompaniment what we would expect it to be?

https://drumbot.glitch.me/ seems to be offline. =(

I think we keeled it with traffic.

or... it's just a glitch!

Sorry, I'll show myself out.

Sorry everyone, I think glitch is having an incident right now! :(

This is awesome. Really looking forward to looking over this source code and finding out about export / MIDI options.

TL;DR: If drumbot.glitch.me doesn't work for you, try running it locally [2]. It's surprisingly effortless!

First attempt at trying this out: 502 Bad Gateway.

Second attempt: I thought I didn't want to mess with Node, so I downloaded Magenta Studio [0] instead. When trying to load the Max patch, my potato promptly lost its will to Live. (Ba-dum tss [1].) Maybe loading a 900MB .amxd when you only have 1GB of free RAM isn't a great idea? Anyway, I'm not keen on risking another reboot today, so I'm putting this off for now.

Third attempt: Localhosting DrumBot [2]. Five minutes ago, I didn't have Node installed (I didn't even recall ever touching Node directly at all), but now (npm install … added 247 packages from 284 contributors … found 4 moderate severity vulnerabilities … npm start) the tensors are flowing already! Whoa, Node is so easy, any idiot can use it — even musicians ;)

I peck in a simple melody:

  Fly me to the moon
  Let me play Fly me to

Oops, this started looping at an unexpected point. I guess I should have stuck to a 4/4 time signature? DrumBot is equally confused and lethargically bangs out wonky shit like it's attempting to cure carbon monoxide poisoning with ketamine. Meet my new band: Tourette's Horse Tranquilizer.

I start over, this time with a proper MIDI keyboard plugged in. Initially this fails: MIDI Off messages trigger notes in the same way as MIDI On messages. This is easily fixed by routing the MIDI through a DAW. (Why? No idea.) I play a standard Boogie riff, except slowed down to 100 BPM (because I can't really play keys; I just have this controller for inputting notes). DrumBot analyses this (100% correctly) as a Chicago Blues turnaround and, apparently knowing that six and two is eight, immediately channels Casey Jones[3].

Dear fellow scholars: what a time to be alive! [4]

Btw. @notwaldorf: I just had a go at routing DrumBot's output back into my DAW and obtained good results by dragging parts of my custom drum kit onto the MIDI notes that seemed most relevant. Then I changed the input and now DrumBot is hitting notes I haven't mapped yet. I guess it's using the standard GM drum map? Implementing that in its entirety would involve setting up over 40 separate instruments :/ But I'm not supposed to go quite that far, right? Is there some documentation telling every monkey how to make DrumBot king of the bongo bong? [5]

[0] https://magenta.tensorflow.org/studio/ [1] https://www.ableton.com/en/live/max-for-live/ [2] https://github.com/magenta/drumbot [3] https://youtu.be/RSTF6GJ23LY [4] https://www.youtube.com/playlist?list=PLujxSBD-JXgnqDD1n-V30... [5] https://youtu.be/cfLIlP-GAmg

npm build does work for this on osx and neither does the website... what a busted app. How did this make it to front of hacker news?

Sorry about that; the app is hosted on glitch and they seem to be having some problems serving apps right now, so it’s not something I can fix :(

It's really not your fault. It's really okay if people don't get instant gratification.

Seems like people are slowly running out of ideas what to do with ML.

Are you kidding me? This is one of the things people have been hoping ML could help with for decades. We're only going to see more music composition and production-related ML applications in the future.

I think they may soon revolutionize workflows for electronic music producers. This is an amazing first step in that direction.

I have been working in this space [0] for a number of years, and I agree, it's becoming extremely promising.

[0] https://sonicmultiplicities.audio

Where may one find good datasets for this sort of research? I'm interested in making AI musicians.

There is no such dataset out there. This way of thinking is a very self-limiting approach.

In my case, I built my own, through 10 years of organic instrumental input for the system to chomp through.

That's how you apply machine learning to the discipline of music.

The skill of making music well enough that I can trust a machine to emulate my style is unfortunately absent in me. Your response is not only patronizing but also loaded with false premise.

It's also apparently wrong: https://magenta.tensorflow.org/datasets

Thanks for your contribution.

That's too bad that it's absent in you. Maybe you should try to be a musician first, before trying to master AI-assisted music-making.

You say that I am applying a false premise and that I am apparently wrong -- but what kind of nonsense have those datasets output that comes even close to what I [0] have?

Also, you strongly misunderstand the value of having to build your own dataset of actual human input to achieve anything good in AI-assisted music-making. Good musical AI does not simply seek to mimic a dataset. That's such a pedantic oversimplification of this noble, difficult task that those of us who actually care about this issue face.

This problem cannot be solved like an object detection problem. You have a lot to learn before you start trying to insult those of us who have devoted much of our lives to the "problem" of AI in music.

This is art, buddy. Stay away from it if you are going to treat it like a mindless AI researcher at some institution.

[0] https://sonicmultiplicities.audio

Maybe I’m a bit oldschool, but until ML becomes self-aware it won’t be able to replace a drummer, mixer or any other kind of musician. Anything related to ML and music which I’ve seen so far basically caused the same two reactions: “WTF? They can do that with ML? This is awesome!” “Ok, to be honest I wouldn’t listen to this if I didn’t knew it was produced by ML”

Yes, there are many use cases here. If we're talking composing something entirely by itself, start to finish, the technology is nowhere near there (at least by my subjective music taste standards). We may not need self-awareness to get there, but we'll probably need something closer to AGI.

But I could think of many ways existing ML could help producers and composers while they're working. They can present possibilities and alternatives that the composer can choose from; they can find interesting combinations and permutations and patterns; they can potentially learn from your workflow and save you time in the future, etc.

I kind of have some ideas of how to develop applications like these. Maybe I should pursue them; although I currently work in a totally unrelated field.

Reply on news.ycombinator.com