Comment by 6gvONxR4sf7o | Hacker News Reader

Comment by 6gvONxR4sf7o | Hacker News Reader

Parent Full thread

Actually working in the field, I hate stuff like this so much. Like, it's a cool project, but the reporting and marketing and hype is all so fucking awful. Imagine if a person had done this instead of a model:

> [I] Completed Beethoven's Unfinished Tenth Symphony

> Now, thanks to [my] work, Beethoven’s vision will come to life.

And so on. Your reaction would probably be "no, this isn't Beethoven's vision, it's some random person's vision." In this case, it's that much more egregious because it's not written by a journalist through a game of telephone. It's actually written in this breathless magical way by the person who built the model.

There needs to be a new fallacy or something: "Appeal to AI." Because otherwise we treat "AI completed Beethoven's vision" as somehow more True or Correct than "I completed Beethoven's vision." And it's much more dangerous when we apply that free pass to things like "AI predicts recidivism." Is it just riding off of how people perceive mathematical correctness or something?

> There needs to be a new fallacy or something: "Appeal to AI."

I applaud and welcome this idea

What pains me is that these are the exact articles that make people outside of the field say "AI fails to deliver", "AI is just a bunch of ifs" or "AI is just snake oil the new winter is coming".

Most actually interesting ML/DL projects are not stuff that the public directly interacts with so you end up with a ton of inconsequential stuff like this article and real progress is just not visible.

I couldn’t agree more. We have very little basis to judge whether the algorithm did Beethoven justice and so all the celebration is really for a non-event. A computer wrote down notes after other notes. Maybe they sound not too dissonant. That’s not news.

This statement feels too strong. An algorithm completed Beethoven's tenth symphony, and it did a passable job, maybe better than what a graduate student could achieve. That's exciting for tool-assisted composition, even if it tells us nothing about Beethoven's historical vision for the symphony.

Agree! This is great and exciting. When people learn to play music composed by someone else, they are experiencing/living to some degree the mind of the composer. And music can fairly be described as sets of repeatable patterns. So if AI can take the entirety of a composer's life's work and discern key patterns, replicate them even, then some aspect of the mind of the composer is reborn. This is immortality. A dedicated grad student could probably do it for one composer; AI can do it for ALL composers. And humanity is all the better for it :)

I don’t think a computer could know what the artist intended to create.

I wonder, how accurate can the computer be at predicting the next note (the first note from the uncompleted section)? Is that an interesting question?

Yes, it's a very interesting question! This how many language models work. You might be interested in reading about a metric called perplexity. [0]

The major caveat is that it's invalid to evaluate on completed, polished work, and then extrapolate to what's practically a napkin note. Another important note about it is that some next token predictions are harder than others. For example, try predicting the next word (the _____) in each of these:

> I really love this Dr Seuss Book called "The Cat in the _____

versus

> I really love this Dr Seuss book called "The Cat in the Hat." _______

[0] https://en.wikipedia.org/wiki/Perplexity

You could also make a model which always responds with the word "hat". If the second sentence is the "creative" portion, it would then prove 100% accurate on your in-sample data, and the next sentence could be "Hat, hat hat." And then people could say, see, this brilliant AI speaks like Dr Seuss!

You could also just string a bunch of random words together and build a model that learned to predict them with total accuracy. Of course if the words are truly random, it would have basically no chance of predicting the next random word from outside the learning set.

The conceit really is that some model got enough important, critical contextual information from the partial symphony to be able to predict with greater than random probability the next portion. On a micro level, one note at a time, maybe. But every note further into the unknown is predicated on the previous ones, so on a macro level it's absurd to think it's "predicting" anything.

Thanks for sharing that term.

A score is so much more complex. I need to dive into the approach Playform took. I notice that team is comprised of experts in a variety of fields, who make a lot of editorial decisions on the output of the AI and then decided the style of the output (scherzo, etc.) and where that particular element might be placed in the score. I wonder what would have happened if they just ignored the AI and completed it using their own compositional knowledge?

Calling the prediction accuracy doesn't seem quite right, but I don't know what term I would use. I created a mobile music notation application that has pretty good accuracy when writing on the score. Some users asked for the ability to predict what might be written next. The first challenge is creating a UI to present the choices, but lets assume that is a solved problem. I trained up models and you can probably guess what happens. It is similar to the demos you have listened to where thirty seconds of on original song are then taken over by AI. Things sound similar and then descend into madness. This is sort of fun but not super useful to a composer. Maybe if you, the composer, had a large corpus used to trained with, the AI could riff on your own ideas.

My experiments were with a single instrument and then I tried using the process on multiple instruments. I would consider the end product pretty much useless.

Users of music notation software, especially those who are able to write out their compositions using notation, would find this feature pretty aggravating. Cool for bizarre demos, pretty useless for making a living.

I do expect to see a lot more activity in this space as we learn more techniques, train bigger models and learn how to keep the AI from going off the rails.

That can be proven though. Keep a work by the author out of the training phase, then see if the model can predict it from an incomplete version.

Otherwise...

This thing has been going on for a long time.

First time some seriously tired to make "computer composer", maybe 1960s?, it passed double blind test flying colors. Professional musicians declared that computer could never compose something like this" about computer made music.

Using learning and using generative grammars has been used to imitate different jazz musicians in 90s.

Some modern composers use computers to generate music, they just tweak parameters until it is how they like it. It's still human made.

I agree that the media is overhyping AI without fully comprehending what it can and cannot do.

At the same time, I believe your criticism is a little off-base. Sure, it's hard to tell it apart from how a person would have finished the symphony. That, however, is precisely the point. That's what's so fascinating about it.

Not working in the field at all, I completely agree! Well said.

Reply on news.ycombinator.com