Show HN: noteZilla - Interactive sheet music
notezilla.io
notezilla.io
Minor bug, Beethoven's 5th was off by a measure for me. (Goldberg variation was spot-on).
Also, right/left arrow keys aren't skipping measures (as the keyboard page says), but are skipping whole sets of measures.
But overall, as a musician, I actually found it rather distracting to watch the line try to follow the notes exactly, as the line constantly sped up and slowed down due to rubato, etc. -- indeed, in particularly fluid pieces, it can even be possible that the right and left hands don't match up precisely in actual performance.
It might actually be more clear, and far less distracting, if you followed the time signature for the beat, and simply highlighted all notes in the current beat within a light-pink rectangle overlay, and then that box would instantly disappear, and the next one appear, upon the next beat.
That way, we can focus with our ears on the actual movement in the music, without being distracted by visual movement, but still have a perfect visual indicator of where we are.
But anyways, great work and great idea! Kudos, well-done!
(just moving by one measure is more easily done by clicking in the score...)
Thanks for correcting that :)
As for Beethoven's 5th, it should self-correct, but I have noticed that in some browsers it doesn't quite work... What browser and hardware are you using? I will look into this further.
I have heard from several others that the current way of moving the cursor isn't optimal. Your solution could work. I'm going to explore a few ways and probably offer some new choices to the user. Your suggested solution would be very easy to implement, so most likely I will add that.
Thanks again!
One minor "bug" / missing feature; I have two monitors at different resolution; when moving the browser (maximised) from my laptop monitor (1440 x 900) to second display (1280 x 1024) the staves are put on a different line to the instrument names. If I refresh the browser on my second monitor then move to my laptop screen all works fine though. The same issue exists if I resize my browser; i.e. currently window.onresize events aren't handled.
Great work though - and nice choice of pieces too.
I do also sometimes pull up some classical sheet music and try to read it along with a recording. So if you get a bigger library that includes pieces I end up being interested in, your site would be useful in that respect. I would enjoy the safety of having the red bar following along to make sure I don’t lose my place.
You said on https://news.ycombinator.com/item?id=6226990 that the left and right arrow keys should move by page, and not by measure, and they’re just mislabeled. I think it would actually be more helpful for the arrow keys to move by measure. That’s what I’m used to in other programs, and I would want to do that more often. I often move back a measure or three back to replay a complicated part I just heard. Moving back a whole page is too much, and using the mouse to click the previous measure is less efficient. You could still keep the keyboard page-moving functions, but bind them to Page Up and Page Down instead.
I’ve used Transcribe! (http://www.seventhstring.com/xscribe/overview.html) before to manually mark up the measures and beats of pieces. You say you have some JavaScript tool that helps you mark up pieces, and I would love to see it, but you might also want to look at Transcribe! and see if that makes marking up pieces easier. Transcribe! uses a proprietary file format, but it’s not too hard to extract the beat times by parsing the text of the file.
Another scrolling method you could try to help people keep their place is showing two rows/systems at once. When you get to the end (right side) of one row, move the bar to the left side of the next row, and then scroll the next row up, displaying the row after that below it. I’m thinking of the effect in this video, which uses Finale Notepad: http://www.youtube.com/watch?v=CxTav4S-4Tk.
The javascript that I use to partition a score is extremely simple - all it does is play the song while I click on the note in the score that is currently being played. This allows the script to "register" the x-coordinate together with the the current position in the audio, at each click.
That results in an array of [x-coordinate, time] pairs, which are then used to control the speed of the cursor movement.
If it makes sense for to let people upload their own pieces (some have asked for that), then that functionality would certainly be made available!
As for the scrolling, I think that it's great for piano pieces (or pieces with just a few parts), and I might consider implementing that also. It wouldn't be possible for larger scores with multiple instruments, but I think it would work better than the current solution for smaller scores.
Finally, great idea regarding the measures / page movements! I will re-map the keys so that page up / down scrolls a whole page, and the left right keys just scroll one measure at at a time. Please look out for that improvement in the next day or two ;)
Hell, you could even color code the instruments subtly with some light shading. Maybe make that a checkbox option.
Well it seems that consensus is that the current page turning is not optimal... Perhaps scrolling is the solution. I will definitely invest some time in that to improve it.
Also, regarding the time and key signatures, I very much agree with you, they need to "stick"...
As for color coding, is the purpose to make it easier to follow a certain instrument?
[edited for clarity]
It would be nice if the key and time signature could remain onscreen as the sheet scrolls.
I think my preferred solution for scrolling would be to have the position marker move until it hits the center, then have the position stay centered while the sheet scrolls until it hits the end. I'm not sure what the correct way to handle repeats in my scenario would be--perhaps have the sheet stop scrolling when it hits a repeat.
As for scrolling, it seems like that is a feature that some people might prefer. I will most likely add that as a feature to it.
Thanks!
Removes the disorienting jolt of having everything just change, but also not always moving and being a distraction
Personally, I honestly appreciate the fact that the score is not scrolling continuously! I will try to have it not jolt though.
Maybe additionally highlight the notes that are currently played. And have you tried scrolling the sheet instead of moving the 'now' indicator? And who are you and why are you doing this? Add an about section.
Yup, I can do the highlighting fairly easily, that's a good idea!
Scrolling is easily doable too, but I'm not sure that's the best way for all situations. I might provide it though, and then let the user choose depending on their situation.
About me: a wannabe classical composer / tech geek ;)
(and in the event people are actually interested in this idea becoming more than just a concept, I would be very happy to provide a more elaborate About section)
Even a "right now this is just a little concept I worked on in my free time, hope you find it useful. Tweet me suggestions @..." would be enough on /about.
Cheers
We've built a similar thing at http://particellissima.net at the Classical Music Hack Day in Vienna.
It's a demo, but you may find some UI ideas interesting, for example synchronization and navigation through a video, flowing score, hiding the instrument parts that are not playing for a while, and assembling a score from small pieces with measure images hosted on multiple subdomains to improve the fetching performance in the browser.
And of course more music would be great!
Interesting demo! I definitely like the hiding of the instruments not playing.... also, lots of people seem to prefer a flowing score.
As for fetching more quickly, yup, also worth addressing.
More music? You can be sure about that ;)
Thanks.
I'll send you a tweet or email so we can stay in touch!
Here's the example I was using: http://notezilla.io/p/bach/air-on-the-g-string
[edit: I agree with crazygringo that it can be distracting if you are worried about the red line being slightly off.]
I will definitely work on getting that line synced up. On my computer (Mac OS X / Chrome), it's perfectly synced, but it seems that that isn't the case for some people.
Thanks for the feedback.
I think Chrome for whatever reason will make it "less off"!
Super nice work though.
Well, post yours and I'll upvote it ;)
noteZilla's well designed btw. It's also beautiful as an observer, but, if I were trying to use it as a performer I'd be thrown off by the varying placeholder speed and page turns [as others mentioned]... maybe that should be your toggle if you want to accomodate that auidence — a 'performance mode' which shows multiple lines and a consistent rhythm with highlights around the active bar.
And, I think I will share even though I'm ashamed atm.
FYI, I slowed the page turns down a little bit... in an effort not to jolt users too much (a first step). If you're reading this, maybe check the site and see if it's any better, and let me know?
Also, thinking about implementing multiple lines as well (just haven't decided how high up it will be in the list of priorities!)
http://pianobooster.sourceforge.net/
http://sourceforge.net/projects/midisheetmusic/
I have been wanting to work on this idea for quite a while in my spare time, particularly to enable scrolling and keeping the signatures in place while the sheet scrolls.
For example, it could play a certain section once, then repeat it, but without sound, while the student plays, and then continue in a loop like that until the student feels comfortable with that particular section.
I'm sure there are many other things that could be done in that realm too, so feel to tweet or email me anything that comes to mind (@notezillaio / notezilla.io@gmail.com)
As for incorporating new pieces, it's definitely technically doable and it seems that that's something people are interested in. There might be certain copyrights issues that have to be ironed out first though... ;)
A few minor things: While jumping to time points works, scrubbing does not. Instead, the notes image gets dragged. You could embed the image as a background-image instead of an <img> to prevent that. Or catch the dragstart event on it and return false (http://stackoverflow.com/questions/4211909/disable-dragging-...). A custom Rails error page or better error handling would also be nice (http://notezilla.io/bla). And there are a few console.log's left (TIME!!!! :)
Did you match the music to notes by hand?
As for scrubbing, it was a conscious decision to keep it fixed for now, but I might consider changing that in future version... and also need to fix status pages, etc.
Yes, I match it by hand, but it's fairly easy to do using some javascript that I wrote...
You might of seen it in the Apple ads
it's $13.99, although to try something similar, I got Beethoven's 9th Symphony for free first http://www.touchpress.com/titles/beethovens9thsymphony/
Please tell me you will have some way to use this to display a score written in some open format. and/or interface with MuseScore or some other software.
Alternately, it would be possible to just upload the pictures of the score directly in .png format onto noteZilla.
Such software usually offer 3 views of the same underlying music data: piano roll, staff/sheet notation, and an event list.
Either way, it would be cool to see a writeup about how you implemented this.
And yes, I might actually write a blog post about this so make sure you have signed up on the site :)
As for Audacity, can it recognize the beats automatically?
It's a highly useful tool for the analysis of music recordings.
I, too, am highly interested in the extent to which matching recordings to scores can be automated, as are a lot of music theorists. I hope you can publish a write-up of the process for syncing the audio and the score.
If there is a significant human spend required, then developing an open tool for community contributions would greatly speed up the creation of a corpus of scores.
As for the write-up however, the process I currently use is very simple (and it's pretty much all "manual labor").
To partition a score into time intervals, I have a couple of lines of javascript that record the time the audio is at, together with where in the score I click (and where I aim to click on the note currently being played).
It doesn't take too much time and effort to partition a score this way actually, unless the piece is slightly complex. Beethoven's 5th took a little more effort than Air for example ;)
Hope this helps for now! And I am planning to publish a more detailed write up of everything, so please stay expect that in the next month or so (easiest to hear about via Twitter or by submitting your email to the site).
Thanks for your interest and suggestions!
I've used a similar process to generate a video with a score: http://www.youtube.com/watch?v=OnoBD_QSskE.
Just checked out your video + followed you on youtube! Looks very cool as well!
I love (classical) music and as a music hobbyist I try to dig more into sheet music for a while. I would also pay for some kind of premium service (maybe great content or listening the sheet music in combination with a famous record).
Anyway, I subscribed to your mailing list, please just keep on rocking.
PS: And yes, I own Logic and could play sheet music in that way. But I do think you folks will prepare something more which a hobbyist can enjoy!
How about you email me at notezilla.io@gmail.com and we can chat more? Would love to learn more about your thoughts and how this could be useful to you.
What I was missing is some indicator showing where exactly in the whole piece I currently am. A simple solution would of course be a timeline but maybe something neat like a minimap with a sliding windows of the current view is possible. Like this one could directly navigate through the whole piece by clicking in the minimap.
Thanks much!
A bug: If it reaches the end, then clicking in the score will move the pointer there and start scrubbing but no sound is played. The play/pause button behavior becomes a bit funky as well.
Kudos to Heroku!
Would it be possible to isolate each instrument? Turn each one on/off?
For pop music, it's definitely doable as long as I can get access to the individual tracks. It's actually something I have been wanting to do, so thanks for the suggestion.
That way you could allow users to upload their own songs.
I'm actually using LilyPond already to create the sheet music, so if I do add the possibility for users to upload their own scores, it would most likely be with LilyPond...
Although the user would still need to partition the score (ie indicate at what time a certain note occurs) if the music was to be synced up to it.
It isn't terribly labor intensive but still requires a little bit of work... so maybe, if that's what people want!
But this is awesome by the way, great job! I'd really love it if user-generated content is supported. I have a bunch of lilypond transcriptions from the Real Book that I'd like to upload.
Just need to be clever about it and make sure it's not too arduous a task for the user to upload something. Please feel free to email me or tweet at notezilla.io@gmail.com / @notezillaio and we can stay in touch / discuss further, would love more input!
Well you could automate it -- simply implement 100% accurate polyphonic pitch detection. That should take, what, a weekend? ;)
Albeit, either way this is a great idea!
"This website lets users to bla bla bla..." (cause og:description).
Just something to note.
Makes me wonder about the genius of the coposer - very insightful.
I can highly recommend it. You need to activate their WebVisor functionality to get access to what I just mentioned.