Rethinking Sheet Music, With A Few Bells And Whistles | Hacker News Reader

Rethinking Sheet Music, With A Few Bells And Whistles

npr.org

51 pointsdangrover42 comments

The main problem with sheet music is copyright law. Its differences across Europe will make anyone's head hurt. If that weren't enough, the established publishing industry is extremely conservative. In my experience the musicians --that is users-- would be the last and least hurdle.

This times a million.

Just for some context, Dan is the guy who built Etude.

A company that launched at TC Disrupt does this: http://tonara.com/

"Think of it as sheet music for the iPad generation. Tonara is an iPad app that can map your keyboard pounding to the right place on the sheet music, and then magically flip the page at the right point.

Married to a store selling sheet music, it’s pretty clear that something interactive like this will do to sheet music what Kindle did to hardback books. TechCrunch’s panel of judges were dubious of the size of the market, but our guess is this company will be quietly have its tip jar filled high as it plays a requiem for sheet music anthologies." - http://www.wired.com/epicenter/2011/09/startups-techcrunch-d...

They also have a nice video showing it in action: http://www.youtube.com/watch?v=HBXJZKTOcpw

As an orchestral violinist, my biggest pain point is that I can't mark bowings into one part and have them appear in the parts for everybody else in the section.

I think a killer feature for this (or a competitor...) would be auto-turning pages using the ambient audio. "While I am playing piano and have no hands I have to deftly change the music, therefore I need another human" is a huge pain point for performing pianists.

I'd think out of the box on this, and just give the user another channel to control the page turning. The most natural choice I could see would be some sort of eye or head tracking. Though I suspect that rather than trying to guess where they are in the sheet music, I'd just give them a visual target off to the right or left that if they gaze upon/turn towards it the page turns forward (or back).

The problem with solutions based on tracking where we are in the music, then inferring the correct action, is that it still encodes assumptions about when the page should be turned, which is great... until it's wrong.

> I'd think out of the box on this, and just give the user another channel to control the page turning. The most natural choice I could see would be some sort of eye or head tracking.

I think the most natural is a pedal, which has actually been implemented.

Grandparent's suggestion of ambient audio seems rather inadequate. Pianists don't turn pages always at the very end of the page. They read ahead, and how much they read ahead depends on the particular passage.

"which has actually been implemented."

Yes, I implicitly added a requirement for myself that I not merely duplicate off-the-shelf tech, because that would be boring. One characteristic of that solution is that it requires additional hardware, whereas head tracking (and maybe eye tracking) could be done with the same setup pictured in the article. But I would totally agree that given a pedal, that will be the superior solution on almost every level. Truly, nothing else speaks with the input clarity and precision of a simple button.

The hurdle for this is that to do it well, you need polyphonic pitch detection, which you could say is a little tricky.

However, I worked on the prototype for a music app that does polyphonic pitch detection for a greater effect than page turning. It listens and follows along note by note as you play your (real) guitar and, as you hit the notes properly, actually uses the master tracks of rock songs to give you a very cool sensation of playing the real song. It's called Rock Prodigy: Guitar and is free with per-song IAPs if you want to try it out.

Following an existing score should be a lot easier than de novo polyphonic transcription. For example, you could almost just track the beat while ignoring all the pitches.

With a rhythm-game type app, you can dictate what the user is supposed to be playing. With a sheet music app, not so much.

People will want to play the first few measures over and over, try something again after messing it up, stop and take it from the top and so on. If you turn the page too soon or scroll the music the user is playing off the screen, the customer reviews are going to look like Armageddon has come.

The only way I see that works understandably and well enough is to recognize when the last measure of the last staff is completed. Even that may be incompatible with some music, but it should work well enough in general.

It's impossible to do that with a lot of non-standard notational music. And even with standard music, "the beat" is annoying to track and changes FAR more than you'd expect. Good musicians are constantly altering the beat in subtle and unsubtle ways.

Exactly. Rhythm quantization is one of those things that no app ever gets right.

I've been working on converting midi/virtual midi input (from a performer) to notation in real time for an iPad notation app. It really is hard - I spent a tremendous amount of time on it, an still plan on improving it indefinitely after the feature ships.

Dude, your app rocks. We tried to get in touch with you a while back with no response.

Thanks. If you want to connect, email me phlee@email.arizona.edu.

Is there even a need for pages anymore? Wouldn't it be nicer to have one long continuous sideways scroll in landscape mode.

Even better if the App was able to match the scrolling speed to the playing speed automatically.

Playing speed changes over time, and the problem with continuous scroll is that musicians often need to look away from the music -- at their hands, feet, at other musicians or the conductor -- and if the music has moved in the meantime, it is disorientating and not useful. That's why a page-based model is still the only way to work, even when you think a scrolling model is more efficient.

Not only that, but a lot of times you memorize different sections by their physical placement on the page. To lose that orientation would make memorization harder.

Another real-life need for pages (and page numbers) is for easy reference in rehearsals. You don't want to wait while 100 people figure out how to scroll to a given point without common reference points (although, yes, bar numbers would work, too).

I also heartily agree with the commenter who noted that continuously-scrolling music would be disorienting in real use.

Tried that with the iPhone version that came out in March last year. In practice it was silly and people liked the iPad version better.

I wonder have you thought of splitting pages in two, and showing the top half of the next page while you are playing through the bottom half of the current page? This would keep pages static. (I wish this could be implemented on my Kindle also, as I often push the button for the next page too early, and then have to turn back again to read the last 3 words).

Here's a much simpler solution: a page-turning pedal: http://www.bilila.com/pagescore_turner

Or http://airturn.com/

Tonara does this.

I'm excited about Tonara, but the app itself seems like a tech demo for that specific technology.

They have yet to experience the pain of a thousand cuts that (that my team and I have gone through) that is working with the music publishing industry.

The thing with these apps is that people who don't really understand them group them all together. "Oh, another music-y app for fancy music people!"

While their app now is not remotely a competitor, if they actually get some real funding, they could pivot in a number of ways to compete against apps like mine, because they've been able to position themselves well in the "music app startup" slot by presenting at TC.

Congrats, Dan! What a great write-up. So glad to see how far this has come.

Leaving aside the practical matters native to the iPad, I think not having an extensive classical selection is one of the best ways you could be dinged. It speaks the app's value and a desire to use the app more. And it leaves the door open for a followup story.

It's not the place of any app to solve the problem of needing a full classical library. Various websites, from Art Song Central to IMSLP to Choral Public Domain Library have already attacked this need with vigor. There are a LOT of good PDFs out there already that can be used right now in forScore without any need to pay Steinway or anyone else for things that are in the public domain already.

http://imslp.org/

http://www.choralwiki.org/wiki/

http://artsongcentral.com/

If all you need are PDFs, you can use any PDF reader. This app does a great deal more than that.

Original story: http://news.ycombinator.org/item?id=3001020

Interseting how many startups tackling music business. I was quite interested in a similar startup named Tonara, which debuted on Techcrunch Disrupt[1] earlier this week.

[1] http://news.cnet.com/8301-19882_3-20105113-250/tonara-puts-s...

Congrats on the acquisition Dan!

It's awesome to see how old school companies are making the transition to compete in the digital era.

Synthesia does a very similar thing with Guitar Hero-like scrolling but uses MIDI (so, no sound recognition, you need a MIDI keyboard).

http://synthesiagame.com/ (no affiliation; happy user)

Most of the things people are hoping and wishing for in this thread have already been solved in the excellent app forScore:

www.forescoreapp.com

I'm a professional classical singer and have used forScore in performance.

If only someone would rethink music notation successfully.

Agreed. I'm currently working on a startup that's focusing pretty heavily on that (http://www.tabrat.com/), and let me tell you, even if somebody were to get it 'exactly right', it would still be quite the uphill battle to get it adopted. That's assuming it's a legitimate re-imagining that deviates from the current system of course, and not some C++ style superset. Nothing wrong with teaching an old dog new tricks (in fact it's what we're planning to do in the meantime), but it wouldn't really be anything new, because that is exactly how music notation evolved to be how it is today (excluding tablature for fretted instruments, which should be considered a good, albeit limited, innovation in it's own right).

The people that use notation the most, and take it the most seriously, are usually the ones that have invested a ton of time learning to sight read and knowing how to properly express their ideas in it; hence they are almost certainly going to be hesitant about switching to a new format. We musicians tend to be a conservative bunch; after our first few years playing/experimenting, we pretty much solidify what we like and hardly ever deviate from it afterwards.

I (a guitarist) for example, only really play 2 brands of strings, I have already made vast generalizations in my head about what amp/guitar/pedal brands I like and don't like (as opposed to judging on a product-by-product basis like I would for most other things), and I don't see that changing soon. Likewise, once you develop your notation style, it would probably be hard to really think of trying something new when you've just come up with a cool riff and get caught up in the heat of the moment trying to build on it.

Although I can't wait to see the day the next evolution in songwriting starts getting widely adopted, I can't say that what we have now is really that bad. Traditional staff sheet music does makes it easy to 'visualize' what a piece sounds like, and that's a feat that even well notated tablature can sometimes struggle to convey.

As for my startup? We just want musicians to be able to precisely jot down something like this on a moment's whim (fx and all): http://www.youtube.com/watch?v=YU52hGo3nNc

Yeah, my degree is in Music Theory/Composition, and while I understand where people are coming from when they complain about music notation, I can't see myself investing any significant amount of time learning an alternative system when I already have the current system down well enough that I can sight-read and sight-sing a vast majority of music.

tl;dr I'm not giving up the ability to look at an orchestral score and have a pretty good idea of how it sounds just to learn that skill all over again.

I agree. The janko keyboard layout for the piano appeals to me because no key is different from another. For instance if you learn one major scale on the janko keyboard you now know them all. Doing something similar for music notation is hard, but would be incredibly useful if done well. See http://musicnotation.org/ for some attempts.

What would you change? I'm curious.

That musicnotation.org site outlines the major issues with traditional music notation. Traditional notation is very piano-centric, making learning to read it on a chromatic instrument more difficult than it needs to be. Nearly all stringed instruments have no distinction between sharp, flat and natural notes.

That being said, there are hundreds of years of tradition behind current notation and that kind of momentum is unlikely to change. Guitar players came up with tablature, which is great except for the lack of rhythmic information in the notation.

I fear it's always going to be something-centric though. I play the clarinet and there's simply no easy way to indicate which fingerings to play short of sketching a clarinet and drawing it out (which is what happens, very schematically).

That said, it is remarkable how easy it is for a pianist to play on sight, whereas for classical guitarists that's usually only a skill you can hope to attain if you major in music. Banjo and ukulele are even harder, because of the re-entrant tunings where your highest note is the topmost one instead of the bottom one, destroying what little correlation there was left between sheet notation and hand/finger position.

Reply on news.ycombinator.com