Comment by enkiv2 | Hacker News Reader

Comment by enkiv2 | Hacker News Reader

Parent Full thread

We're quite lucky that neural nets are overkill for procedural music generation.

By that I mean, we have huge masses of music theory, applied across genres and focusing on differences between genres, in terms of heuristics for both analysis & composition, and because of a tradition of procedural generation of music that goes back a couple centuries, a lot of it is fairly easy to translate into computer programs. (For instance, end-to-end serialist composition is easier for computers than it is for people, while canons and other mechanisms for creating permutations of melodies are equally straightforward.)

This doesn't translate into a straightforward method for putting in two wav files and producing a third with transferred style, but it does mean that a sufficiently motivated person can write something that translates notes between two known genres with greater ease than they would with images.

(Text is somewhere in the middle. I've worked on a couple attempts at 'style transfer' for text -- mostly using word2vec.)

I think you’re overestimating the power of music theory to serve as a basis for generation or modification of music, and also the scope of music that it explains. Rhythm has been central over the past 100 years since Western pop music has a lot of its roots in American blues. However there is surprisingly little that music theory has to say about rhythm or groove.

There are a lot of regularities and patterns in harmony and harmonic sequences which music theory covers, but there are also a combinatorial explosion of melodies that will be justified by music theory in a particular harmonic context. The choice of which melodic path to go down is very poorly constrained by music theory.

Agreed, although I generally think in terms of "obeying traditional music theory (to some extent) is necessary but not sufficient for a listenable melody". This also changes depending on genre and era - one reason for targeting early Western music, such as early two or 3 voice counterpoint is that composition was more regular and in accordance with theory (which was kind of codified after the fact, focused on explaining these types of composition), though the "reward problem" remains. Coming around to free jazz or extremely "modern" composition means violation of most or all rules while still being "musical" (at least to fans of those genres) - that is going to be even tougher and we are pretty far from automated generation of long multi-part composition without some extra hints from theory built into the models and data, even for music that closely follows theory.

This is one reason many people in the space are focused on new tools for creators, instead of "automated creation" - what is "cool/interesting/listenable" is ill-defined, but making something which allows creators to explore the sonic landscape in really different ways seems a lot more plausible to me - more "weird synthesizers with neural networks" than "robo-artists".

Theory turns out to be nowhere close to sufficient for defining musicality.

If you write a rule-based counterpoint solver - this has been done, with varying levels of success - you'll find that not only do you have to include an extra set of rules that aren't defined in any of the standard texts, but that the best output you can expect is musically mediocre.

The other approach is to create a patchwork of idiomatic cliches. That usually sounds more convincing, but still isn't musically interesting. And you can usually hear where the edges are glued together.

It turns out that there is no theory of "good" music. It literally doesn't exist. All the standard texts for each style - and it doesn't matter which style you pick, from Palestrina to pop - are very incomplete guides that rely on human intelligence and creativity to fill in the gaps.

Throwing neural networks at this problem doesn't make it any easier, because no one knows what to look for.

Features in artistic style transfer are easy to parse - shape, texture, and that's pretty much it, all packaged in a ready-to-go 2D distribution.

What are the musical features that define not just one possible musical style, but all of them, and would allow anyone to morph smoothly from Wagner to Taylor Swift to Balinese Gamelan to DubStep?

On rule based solvers - this is true if you simply stop at "be within the rules" and do strict constraint solving ala the coloring problem.

I think there is potential to do something by blending all 3 (rules and constraint checking, rewards (perhaps based on idioms, riffs, and cliches), and some additional discriminator/critic/metric learning) if you narrow the criteria sufficiently (3 voice counterpoint in the style of Josquin de Prez for example).

I would really be pleased to see something that can fill out "theory 101" counterpoint worksheets against a cantus firmus, even if the results would be poorly graded on "style" it is someplace to start getting feedback and collecting data to try and quantify that it factor for a narrow narrow subsection of the wide world of music.

I have a very weak grasp of music theory and write a lot of music generators based on that weak grasp. They are mostly successful, compared to my prose generators.

It's not so much that I'm overestimating music theory, but that the state of really concrete analytic structures in other arts (in terms of being well-positioned to produce new works from PRNG output) is pretty bad.

It's pretty straightforward to construct a program that produces mediocre music that really resembles human-written mediocre music. Computers are also really good at producing poetry that is indistinguishable from the work of mediocre human poets. Computers are not good at producing images that look like they were hand-drawn by a 12 year old Deviantart user or producing stories that read like they were taken from fanfiction.net

Generating rhythms is a problem that has been solved by arpeggiators, step sequencers, analog modular rigs and more sophisticated tools like KARMA. You don't need machine learning for it.

This is a greatly oversimplified view of rhythm. There are many rhythms and grooves that do not lock in with "the grid". These tools will most likely not produce a natural pattern of velocity that sounds appropriate for the generated rhythm. Step sequencers are a tool for inputting rhythm, not for generating it. Arpeggiators typically have a consistent rhythm (hitting on every one of some subdivision).

Modern step sequencers (for example, Elektron boxes) are way more than just a grid. They have microtiming, parametrized triggers, parameter sliding, probabilistic and conditional triggers, and are capable of running multiple patterns of varying lengths that reset at different rates.

Ok then tell me how to generate a drum part 30% of the way from Buddy Rich to Neil Peart.

I think the issue with music theory is its actually an after thought. A bunch of guidelines to reproduce something that is often first developed by simply pleasing subjective taste. Example: Tonal harmony didn't really lend itself to 'blues' Idom7-IVdom7-Vdom7 but the later jazz harmony was developed to accommodate it.

Reply on news.ycombinator.com