Comment by jacquesm | Hacker News Reader

Comment by jacquesm | Hacker News Reader

Parent Full thread

That's an interesting comment. So, the midi->score conversion is indeed one of the harder parts, I spent a lot of time on that and I'm not yet satisfied with the results. There are quite a few ways in which this can be improved further, there is a long list of 'todo's' where I basically have worked out how to make the improvement but I have not yet gotten around to actually implementing it.

Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs. You then have to do all kinds of tricks to even begin to approach what the score looked originally. Lots of enharmonically (identically sounding) combinations may have looked completely different on the original score, and when you start adding the various chord combinations it gets even more complicated. Even so, I'm strongly convinced it can be solved and we have made considerable progress on this already.

The 'wiggle room' aspect of it has been addressed to some extent. For instance, we build up a map of all the notes that are sounding at a given time, and will ignore notes that are technically 'wrong' but that are still sounding. This allows for much smoother restarts after mistakes.

Your final point is a bit tricky, but thank you for the hint, I will definitely add it to the list of things to look at to see if we can find a way to make that work.

Musescore does do a great job and is open source, so there's that. Another option is to have users manually upload midi with a proper score attached to some store or community repo with user reviews. It's less satisfying as a solution, but it means all it takes is one motivated user for everyone to enjoy the score.

For my final point, you can model this as a hidden markov process. You're observing notes, but not the user moving his mental "cursor". Classical filtering algorithms should do well here.

Ah, one more missing feature, good scores have fingering, but it gets lost in the midi. I'm pretty sure fingering can be infered though and this would be pretty powerful. Amusingly, it can also be seen as a filtering problem:

- At any given point, a hand can have a finite number of fingerings, and that number isn't that large. Give each of them a difficulty.

- Now, make that difficulty conditional on the previous hand position.

- Run Viterbi's algorithm on the score to minimize the difficulty.

The difficulty can be obtained from an expert or through some physiological model, but they could also be inferred from an existing body of music sheets.

Fingering we have a pretty good hold on already, so I'm not too worried there.

Importing alternative formats (xml for instance) is definitely an option worth looking into, but because the bulk of the material that is out there that people can freely access is in midi format we chose that one first.

> have users manually upload midi with a proper score attached to some store or community repo with user reviews. It's less satisfying as a solution, but it means all it takes is one motivated user for everyone to enjoy the score.

Musescore already has such a collection of sheet music https://musescore.com/hub/piano so it'd also be possible to allow users to import from there, going from sheet music to midi instead of trying to reverse the process.

Yes, but then you would still lose quite a bit of information. Besides that, musescore is pulling a bait-and-switch on the user generated content that makes gracenote look good.

Piano Marvel allows you to add your own pieces to the library, I did it with one I found online. What they recommend is using Musescore to convert the MIDI to a score, fixing it up in Musescore and exporting as MusicXML. Musescore did a creditable but not perfect job of the conversion, but fixing it up was easy. Then you import both files into Piano Marvel. I'm actually not sure why they need the MIDI at all once you have the MusicXML, given that Musescore can play the music from the score directly.

> Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs.

As using MIDI will probably never be perfect, have you thought about using the score in a suitable format, like LilyPond, directly?

Yes, absolutely, and we will add more import formats.

I don't really follow the idea that "midi is lossy" in the sense described. Because it uses note-on/note-off pairs? I don't see the problem there. Certainly the idea that a score somehow has more information in it than a midi recording is ridiculous. A score always requires interpretation. The midi is, in contrast, fully specified, very little room for interpretation.

A midi recording contains pretty precise velocity info for each note (0-127) that is either not present at all in a score, or is present only in the form of a limited number of dynamic markings[1], which govern the flow of dynamics imprecisely, and almost not at all on a per-note basis.

With tempo it is similar to dynamics: midi will precisely determine when notes turn on and off, down to the millisecond, in the process resulting in the midi performance having a precise tempo. In contrast, musical scores have limited means to indicate proper tempo[2], necessitating (as with dynamic markings in a musical score) much interpretation by the musician.

All of this is not to say that a midi performance is somehow "better" than a score. There are good midi recordings and bad ones. A good recording will generally be made by an accomplished musician who in making the recording records fairly precisely a particular (and good) interpretation of the score. Likewise, a bad recording would generally be the result of a poor interpretation of a score. But whatever the case, the midi recording is far more precise than the score itself.

[1] https://www.aboutmusictheory.com/music-dynamics.html

[2] https://en.wikipedia.org/wiki/Tempo#Basic_tempo_markings

> Certainly the idea that a score somehow has more information in it than a midi recording is ridiculous

A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.

Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?

When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).

Thanks, yes, I would agree that a midi recording and a score to a large extent encode different types of information. The score has less of some of the kinds of information that are in fact present in midi. And the midi, of course, doesn't specify anything about ranges of interpretation; it simply represents a specific interpretation of a score.

Midi note code 61: is that a C# or a Db, you pick...

Reply on news.ycombinator.com