Pianojacq, an easy way to learn to play the piano
jacquesmattheij.com
jacquesmattheij.com
On the off-chance that this will inspire the article's author for their development, or simply for the curious, this has been my experience with it:
The primary mode of display is "Guitar Hero" style (it's hard to describe just look at this video to get an idea[2]) but it's possible to display the sheet music. It lets you practice and grades you depending on how close you're sticking to the score. It's very nice to be able to practice one hand separately while having the app play the other hand. I do fairly well reading sheet music but the first time deciphering it can be a bit tedious, using the app speeds this up considerably for me.
Synthesia is almost great but it has a lot of small shortcomings which add up to a bad user experience, for example:
- The conversion from MIDI to a readable score is very poor compared to, say, Musescore. The sharps and flats are generally off (e.g. randomly a double flat D will appear instead of a C, even though the score has a C, Musescore interprets the midi as a C, etc).
- Not enough wiggle room when playing, e.g. if a chord is not arpeggiated in the midi file, you won't be able to move forward by arpeggiating.
- The app really depends on you following the score exactly and linearly. It should also let the score follow you. For example, if you're playing at a given pace, miss, stop and repeat a bar, the app should detect that automatically instead of just carrying on expecting you to keep playing.
Midi is 'lossy' in the sense that it flattens a bunch of knowledge about how a piece is to be rendered into note-on/note-off pairs. You then have to do all kinds of tricks to even begin to approach what the score looked originally. Lots of enharmonically (identically sounding) combinations may have looked completely different on the original score, and when you start adding the various chord combinations it gets even more complicated. Even so, I'm strongly convinced it can be solved and we have made considerable progress on this already.
The 'wiggle room' aspect of it has been addressed to some extent. For instance, we build up a map of all the notes that are sounding at a given time, and will ignore notes that are technically 'wrong' but that are still sounding. This allows for much smoother restarts after mistakes.
Your final point is a bit tricky, but thank you for the hint, I will definitely add it to the list of things to look at to see if we can find a way to make that work.
For my final point, you can model this as a hidden markov process. You're observing notes, but not the user moving his mental "cursor". Classical filtering algorithms should do well here.
Ah, one more missing feature, good scores have fingering, but it gets lost in the midi. I'm pretty sure fingering can be infered though and this would be pretty powerful. Amusingly, it can also be seen as a filtering problem:
- At any given point, a hand can have a finite number of fingerings, and that number isn't that large. Give each of them a difficulty.
- Now, make that difficulty conditional on the previous hand position.
- Run Viterbi's algorithm on the score to minimize the difficulty.
The difficulty can be obtained from an expert or through some physiological model, but they could also be inferred from an existing body of music sheets.
Importing alternative formats (xml for instance) is definitely an option worth looking into, but because the bulk of the material that is out there that people can freely access is in midi format we chose that one first.
Musescore already has such a collection of sheet music https://musescore.com/hub/piano so it'd also be possible to allow users to import from there, going from sheet music to midi instead of trying to reverse the process.
As using MIDI will probably never be perfect, have you thought about using the score in a suitable format, like LilyPond, directly?
A midi recording contains pretty precise velocity info for each note (0-127) that is either not present at all in a score, or is present only in the form of a limited number of dynamic markings[1], which govern the flow of dynamics imprecisely, and almost not at all on a per-note basis.
With tempo it is similar to dynamics: midi will precisely determine when notes turn on and off, down to the millisecond, in the process resulting in the midi performance having a precise tempo. In contrast, musical scores have limited means to indicate proper tempo[2], necessitating (as with dynamic markings in a musical score) much interpretation by the musician.
All of this is not to say that a midi performance is somehow "better" than a score. There are good midi recordings and bad ones. A good recording will generally be made by an accomplished musician who in making the recording records fairly precisely a particular (and good) interpretation of the score. Likewise, a bad recording would generally be the result of a poor interpretation of a score. But whatever the case, the midi recording is far more precise than the score itself.
[1] https://www.aboutmusictheory.com/music-dynamics.html
[2] https://en.wikipedia.org/wiki/Tempo#Basic_tempo_markings
A midi file is designed to contain different kinds of information than a score, so it can absolutely contain less information (of a certain kind) than a score.
Imagine a group of notes that are attacked very close together in time in a midi file - is it meant to be a chord played without arpeggiation? A chord arpeggiated as fast as possible? A chord arpeggiated to a specific tempo?
When a composer writes (say) a chord with an arpeggiation sign next to it, it specifies that the notes are to be played in a certain order, and quickly. There are multiple possible performances that comply with this "specification". MIDI cannot encode this information (the information being the set of acceptable performances).
The way I learn is by bars, repeatedly re-playing bars say, 1 to 4, until I get them, them 3 to 7, until I get them, etc.. But most piano apps want you to start playing from start to end slow, then faster, etc.
So: piano apps should allow breaking a piece into bars, allow you to repeat those bars again and again until you exceed certain score. Maybe even keep track of which bars are harder to play, and repeating the difficult bars more often, etc.
(Not affiliated, just something I personally use)
I'm a piano noob but wouldn't you want to practice catching back up with the beat if you ever intend to perform?
I'm not quite happy with Synthesia. There is no visual feedback on completing a note. This has been a staple feature in Doremi Mania and Keyboard Mania for twenty years, so they have zero excuse for not implementing it.
However, for me what made piano a whole lot more fun was ditching the sheet music and opting to learn about chords, all the different ways to voice them, and the various scales that fit well on top of them. I started taking lessons with a pro jazz pianist (instead of a more typical classically-focused piano teacher), I gave up on sheet music altogether and started working off lead sheets instead (just chord symbols and a melody line). I am so pleased I made that decision, it's so much more satisfying playing a tune in your own way rather than just aiming for a note-for-note reproduction of what is written on a sheet.
I still have loads more to learn, but I'm fairly pleased with where I've got to so far. I've got a few videos on youtube so you can judge for yourself if interested: https://www.youtube.com/channel/UC_NWuVoCn-Oc1KIiHEhe7Eg
However, although I think I can play reasonably well now, my sight-reading is awful, and it does hold me back because it makes it a lot harder for me to learn new techniques from sheet music (which I am actually interested in doing, as opposed to simply learning the sheet music note for note). I can do it just about, but it's painfully slow, so I am usually too lazy to bother.
Given the choice of either playing with full sheet-music or learning chords/voicings/scales and how to put them together, I'd pick the latter no question, and that is what I recommend to other adults who are learning piano, but I do think it's best to have both. So I can definitely see the value in this, even if it doesn't currently seem to teach the theory and improvisation that, in my opinion, is what really brings the joy into piano playing. I might give it a proper go myself some time soon, to see if it can help me improve my rubbish sight reading.
I can also imagine this being very useful for a beginner... Although I'm singing the praises of chord theory and improvisation I'm guessing most beginners might realistically do better starting with sheet music, for a while at least.
In summary: it might not teach everything, but it looks really useful nonetheless :)
The conventional way of learning piano has a student learning sheet-music pieces that get increasingly difficult as the student improves. It's a fairly linear path, and the vast majority of students will never be able to play a difficult piece like, say, Chopin's Fantaisie Impromptu, as well as a top performer.
Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Composing your own music takes it a step further again.
Obviously there are going to be incredible performers at the high end but there is plenty of fun to be had in the middle and even on the low end.
I have a few pieces that are so beautiful compared to evertyhing else out there that if that were the norm those pieces might as well never be performed again, but then again, it's all subjective and what I like you might not.
> Similarly the vast majority of students going the chords/voicings/improvisation route will never be able to play Over The Rainbow as well as Keith Jarrett, but they can at least have the satisfaction and enjoyment of doing it in their own unique way.
Yes, true. And that too is a lot of fun. I did some jam sessions with the sax (and I wasn't all that good), most fun I had making music to date.
> Composing your own music takes it a step further again.
That might actually at some level be more essential than just reproduction. The cave men had it easy: everything they did was original.
The beauty of a piece doesn't depend on its technical difficulty. Back when I used to play classical music more I loved playing a few of Chopin's Nocturnes repeatedly - they were about the right level for me so it was fun, and I could play them pretty well, I think. But I also spent a lot of time learning a few harder pieces (like Fantaisie Impromptu), and I remember getting frustrated that it took me ages to learn them and, although in the end I could play them sortof OKish, my efforts just weren't a patch on what a better pianist could do.
Looking back I can see that I shouldn't have picked such hard pieces until I was really ready for them. It was probably a big part of why I ended up quitting for quite a few years before taking it up again more recently with an entirely different approach.
Give it a try if you have an opportunity, I'm curious what you make of it given that you are roughly on the same path but clearly very much ahead of me.
And I might as well put in a feature idea (which I suspect would be rather tricky to implement but here goes anyway!): it would be awesome if it could take input from a mic and figure out the notes from that, so it could work with acoustic instruments too :)
I've not taken the plunge yet, but I was convinced this was the way to learn by Scott the Piano guy infomercial on TV. https://youtu.be/k50uk8elR0M
Some day...
One thing I did find was the very helpful, was putting those stickers with note names on each note on the keyboard. They help me keep track of what I'm playing, and when I compose something I rely on the stickers to help me transcribe.
I find complex rhythms particularly hard to read. Figuring out timing from a sheet without hearing it is like pulling teeth :D That said, listening to a recording of a great pianist and trying to work out exactly what they are doing can also be very difficult and frustrating when there are chords and complex patterns in two hands, so a good transcription can help a lot. I have a book of Bill Evans transcriptions that I would definitely do more with if I were better at reading music, or had software assistance!
https://github.com/apankrat/note-detector
It's not without issues still, but tends to do a reasonably good job at what it aims to do.
For what it's worth, the auto-correlation route is simpler and more reliable. FFT alone is not enough, because the spectrum tends to be rather ambiguous. There's still a pattern to it for every note played, so it should be possible to deduce a note with a neural net. But I just didn't have time for that, maybe later.
PS. Give my thingy a try if you have a moment. I'm curious if it actually works with any pianos other than mine.
I will definitely use your software. What would be great is to have a 'virtual midi device'. If it works well enough are you open to licensing it? We will sooner or later hit this problem and such a component would make our lives a lot easier, it would also allow me to practice on my real piano using the software, which if I didn't want to play the piano I'd give my left arm for.
On another note: when you know what should be played it is actually easier: you only need to output a '1' or a '0', what is played is what is expected vs what is played is not what is expected.
This may be a simpler problem to solve, and in that context it might even be able to detect chords.
Re: the another note - yes, indeed. I too realized this, because mis-detected notes are usually off by a full octave.
As a side note, after going reading through various papers on the subject, it would seem that an ultimate note detector should really be based on a Const-Q spectrum analyzer feeding into an NN. Training the net, even from scratch, can be as simple as sitting in front of a piano and then repeatedly playing each note in all possible ways. Then, throw this detector in a group with waveform-based detectors, get a weighted consensus and that should, in theory, be pretty damn accurate.
I know how to read music and started taking classical lessons as an adult. Some time last year a friend recommended me to check out Melodics. It has 3 instruments: keys, pads (finger drumming), drums. I've played pads for few mont;hs, that was good, I've felt it improved my timing and rhythm skills quite a bit. Then I switched to keys. I didn't really like that; it felt like I'm mindlessly repeating whatever is on the screen instead of internalising music. OTOH, when playing pads it felt that something stays in my head after playing. So I guess this may be useful to some people.
For me it'd be more interesting to have some sort of memory trainer - I play something on screen few times, then I replay from memory and app shows me a diff maybe? Or when I'm learning a piece, it would give me a challenge "play bars 21-25 both hands" or "play bars 19-23 left hand only".
[1] https://melodics.com/, I'm not affiliated in any way
>The software is not ‘open source’, we have not yet decided on how we plan to go forward with this project in the longer term, but the code is on GitLab and if you want to see what makes it tick or suggest improvements then of course you are more than welcome to do so.
It does exactly that, pick 'auto' mode.
I just opened this and spent 20 minutes trying to play 4 bars I wrote a year ago in MuseScore. I am definitely a noob at playing, hahaha. I always figured being able to play in real-time isn't of much value, but I'd like to be able to improvise and use a piano to enter notes when composing at least, and then I saw some of Lionel Yu's videos and bought a keyboard... (https://www.youtube.com/c/LionelYuPiano/videos)
I definitely appreciate the auto-replay, simultaneous note count limit, waiting until you hit a key (both at the start and mid-song if you're slow), and scoring.
Another piano app, by me, this is as rudimentary as one can get, in Python. It turns your PC keyboard into a "piano":
Play the piano on your computer with Python:
https://jugad2.blogspot.com/2013/04/play-piano-on-your-compu...
The comments by people wih more music knowledge than me (zero) were interesting and informative.
Attention Mozilla
Bravo! Well done and well said - I really wish more projects and systems were developed with this mindset, viewing users as people to be empowered rather than as resources to be captured and exploited.
So you should probably add something like some introduction material / tutorial... maybe a couple of videos.
This project is a lot of work as it is today, it was made mostly to satisfy a need that I felt myself and my colleagues picked it up and ran with it so now it has some proper power behind it. We fully intend to 'stay the course' and make this a best-in-class offer but there is a long way to go before we can claim that we have achieved that.
As for the satisfaction, I think you may be off the mark there, I've had enough piano education to know what that was like and Luca is having a lot more fun with this program than I ever did being taught the piano by teachers at roughly the same age.
Note this 8(!) year old bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=836897
It's embarrassing.
That is misleading. Mozilla chose not to implement it for security concerns. More context: https://github.com/mozilla/standards-positions/issues/58#iss...
Meanwhile, there are 100's of WebMIDI capable applications now and Mozilla still claims there is no demand for a feature they do not support. That's backwards, of course a feature that you do not support is not in demand because people simply go elsewhere.
The cumulative effect of a lot of little crappy decisions like this is part of why FireFox' market share eroded (the other is Google's anti competitive behavior).
Will you be able to improvise after enough of this? Play songs by ear?
There are a few fundamental musical concepts that I'm not sure you easily pick up by rote memorization:
* Most music is based around "chords", which have non-obvious mappings to physical keys. (They're linear intervals in frequency space, but that doesn't make it easy to realize that D/F#/A and Ab/C/Eb are two representations of essentially the same thing)
* Likewise, songs can be transposed up or down without affecting their fundamental attributes. (They just sound "higher" or "lower"). However, to do that, you need to be able to shift +/- N keys which is tricky given the difficult layout of a piano. To really do this well, you need to understand the concept of scales/modes.
As for improvisation and playing songs by ear: improv requires some knowledge about how music is structured, I have some ideas on showing what the underlying structure is but these have not yet been implemented, we are for now concentrating on accuracy of the rendering of the score (which is still plenty of work) and fingering.
Playing by ear is mostly to have a good association between the keys and the sounds they make, this gets harder with chords. As with everything, practice makes perfect and doing this frequently seems to help me quite a bit by picking out songs by ear. So even if it isn't a direct goal of any of the modes of operation of the software it does seem to be a side effect. We will add an 'ear training' module at some point.
Since you're rendering the score in realtime, you could let users transpose songs up and down which would really help them learn scales.
Very nice project, though.
This and the way it shows these little lines when you hit some wrong keys is really helpful if you are just learning sight reading. Credit for this should go to Piano Booster probably though.
How many Herbie Hancock's have you come across after finishing a grade 1 piano book? Of course a single exercise app or book isn't going to create the perfectly rounded musician. For a child or beginner player, this looks like a great tool.
Advanced musical skill is entirely composable. Having solid improvisational skill or technical ability largely depends on how much you practice those various facets of your instrument. The reality is that probably 95%+ of people who pick up an instrument in their lives never make it to that point anyway.
You're trying to critique this tool with the expectation that a single learning resource should address all aspects of playing an instrument, when that's never been true of any musical learning resource ever.
I'm not sure it's quite beginner level, as I don't see anything that shows you where the note is on the keyboard? You'd have to start with a bit of reading knowledge, or be willing to just hit random keys until you figure out the mappings.
So no cynical HN commentary from me! If it works, it is definitely a big help for learning to play piano.
The timing/spacing of the notes is a bit off though, I wish it scrolled according to beats instead of speeding up and down depending on what type of note was being played. I understand it has to stop and wait for the note to be played, but some of the whole notes threw me off.
He does two different "tracks" according to what the student wants: (1) learn currently popular songs or (2) learn didactic music in the classical tradition (e.g. Czerny). In either track he's really teaching music theory: most popular music will be I-IV-V; jazzy music with turnarounds do ii-V-I tricks, etc. He doesn't teach an instrument, he teaches music.
We would very much like to teach things properly but for that we need more understanding and someone who has been teaching music would be a huge asset to us.
If you could please ask that would be awesome.
Curiously, my audio memory is very strong. I can hear music in my head clearly and in detail, even songs and pieces I haven't heard in years. This does not translate to "finger-memory".
But I do go through spurts, where I practice every day but then for whatever reason step away from days or 1-2 weeks, which might be killing the neuron connections.
I also round-robin my nighttime between piano, guitar, and composition, which probably ain't helping.
I also play piano, guitar, and compose, and I used to struggle with memorizing, but I've been reading a lot of piano practicing theory and have drastically upgraded my finger memory, mostly by using these resources:
https://www.pianostreet.com/smf/index.php?topic=12590.0 (look for Bernhard's post)
Loosely, stop using sheet music except when you're initially memorizing the piece, learn music in much smaller chunks than initially expected, and play entirely from memory whenever possible, even when you're practicing the repetition of something.
The fact that you can remember music easily in your head means that if you spend 1-3 years really banging out the hand-brain connection, you might be able to rocket your playing forward. Raw improvisation where you think of the melody before you play it will help, too, if you don't already do that.
It'd be amazing if it's not a memory problem (at least, not any more so than for most other people who learn piano) but just have to learn a new way of connecting them neurons.
Is that correct?
Another frustrating thing is it's not just long-term finger memory that kills me, my real-time memory bus is slow when learning a piece -- the transfer of "what notes comes next" down to my fingers, just very slow and error-prone. I've tried a mental exercise where I try to only think about what to play "next" and just let what I'm playing "now" take care of itself. It kind of works and feels very different. Basically, to stay 2 beats ahead of the music at all times.
Ugh, but it's still a struggle. My finger dexterity is fine. My musical knowledge is actually pretty advanced and I can play expressively and convincingly -- but only when I can remember the damn piece. My memory is like a balloon that won't stay inflated.
https://www.amazon.com/Classics-Moderns-Intermediate-Grade-M...
Some parts of what you write remind me of stuff I've read about flying an airplane: never let the airplane get ahead of you. In music that would be 'never let the music get ahead of you', once it does the cognitive overload becomes overwhelming.
Then: I listen a lot to the pieces that I want to play, over and over again, until I have the music in my head and then playing piano becomes a lot more like singing, instead of memorizing the movements you memorize the music and playing then becomes actual playing, you now have time and braincycles to spend on making it sound nice rather than just what to play and when. I only have a few pieces in my very limited repertoire at that level but the difference is huge with the remainder, I can make them sound in whatever way I want, I can vary them when and where I want to play variations on the entire piece by changing style and so on. That is what playing piano to me is all about and I intend to spend a lot of time on reaching that kind of proficiency with more music to broaden the range a bit.
I wished I had some tip to help you, also, I'd love it if you gave the software a spin and let me know if there is a way that I can improve it so that it does help you, if that requires setting up a trial version then I'm all for it. Let me know if you're up for this. best regards, Jacques
I think I'm finally gonna try something I've been thinking for years: Photocopy a score, then take scissors to it and come up with a condensed version with only the parts that I blank on. I might discover that in a four-page piece, I only need a half-dozen nudges.
Thanks for the thoughts, Jacques!!
I'd prefer something small- only two octaves and a plus/minus button to shift octaves.
While you're right about onboarding effort, my guess/hope is that the required driver software shouldn't be hard for me or manufacturer to write.
It looks like your site isn't detecting whether the webmidi feature is present though, it looks like its just doing dumb UA filtering.
So that must be something else.
You can check the code if you don't believe me, it's plain JS.
I thought I’d read that WebMIDI was declined to be implemented by Apple because of security issues (including the ability for random webpages to send firmware updates to connected MIDI peripherals), and going back about four months I found an article here on HN (https://news.ycombinator.com/item?id=23676109 — note the top comment https://news.ycombinator.com/item?id=23679063).
I seriously doubt that Firefox is going to implement this.
It's typical Mozilla. We won't do 'X' because 'Y' but when you drill down they could easily do 'X' but they won't and 'Y' is just whatever they can throw up to get away from it without having to dig in and get it over with in the best possible way. If users want something and you start throwing up technical reasons why you won't instead of can't then you've lost that user if alternatives are available.
This is how you kill your market share.
An interesting bit from one of my favorite IT related book, the Soul of a new machine revolves around the 'mode' bit. The engineers all want a 'mode' bit because it will make their life that much easier, and they keep throwing up flak why it has to be that way and the CEO keeps his foot down and says 'no mode bit, period'. 'I don't care how you get it done but that's the constraint and you will have to deal with that constraint'. And so in the end they find a way.
With all the smarts at Mozilla if they really wanted to get it done, I'm sure they'd find a way. The existence proof is Chrome, and I've yet to hear of a random website corrupting a synth's firmware using Chrome.
Piano sheet music includes these finger position hints. Unfortunately they're not optional.
It's hard to include correct fingering in the model without some kind of camera and image recognition system, or special keys and finger sensors, or something equally obscure.
There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well. The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
Acquiring (and maintaining) technique requires hard work and time, but it's relatively easy once you combine these two factors. Developing a good ear, a good sense of rhythm, when to play on tempo, slightly before the tempo or slighly after, really understanding the music, that is infinitely more difficult so it make sense for a teacher to stress that part.
You're not joking either! I played trumpet when I was in MS/HS. Sure, you can play simple songs at your first concert after 3 months of playing - but it takes 3-5 years to get competent, not an expert mind you, but competent.
I started playing guitar as an older adult. Same story. You can start playing songs starting with lesson 1. This is great! Look at all the progress I'm making! I'm on year 7 now and I'm just starting to get to the point where I'd say yeah, I'm good. Not great, but good.
To your point, it takes a long time to master an instrument and no one has discovered a short cut. It's hard work.
On another note, once you appreciate the work it takes to not just become good but become great at playing an instrument then support those who've dedicated their lives to becoming great. Otherwise it's a skill humanity is going to lose over time and that would be really sad.
Part of the art of practicing is the mental work of abstraction and internalization of the music - breaking it into smaller sections, understanding the key changes, seeing the broader rhythmic structure, etc. - if you don't have this and (more importantly) don't have a teacher to help guide you through this process, you're definitely going to hit a brick wall once things start to get woolly.
They actually do.
What they don't take into account is whether a particular black key is a sharp or a flat, and this causes all kinds of headaches. https://en.wikipedia.org/wiki/Enharmonic
It's the second bar on, the left hand goes A F# B# D# B# F# - Aflat F# C Eflat C F#
Only the first note has actually changed though.
Enharmonics in the same bar for the same pattern. I suppose there's some musicological explanation for this, maybe something along the lines of working towards a key change, but good luck intuiting this in software...
They can describe time signatures and tempo.
And yes, they don't handle enharmonics because in MIDI, notes are known only by number, not by any name given to them by any part of any given musical culture or theory.
And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
I know the scale/key/mode as such as not encoded in the note on/off signals but once you know what key you are in (and this can be determined in a variety of ways, including from the statistical properties of the note on/note off signals) the problem becomes a bit more tractable. Still, you will never get it perfect and that is not the goal, the goal is to get it as good as possible (and we are not even there yet).
> And because of the MIDI Tuning Standard, you cannot actually say a priori which notes (i.e frequency) a given note number corresponds to, although in practice most things will take the note number and map it to a a 12TET 440Hz standard.
This is true, but for a teaching application like this one it is simply not relevant.
For your purposes, it may be enough, but in more general terms, it's woefully underspecified.
A better way for MIDI to have done this would have been to use a "standard" scale numbering system combined with a tonicity count. An example or two is given here:
https://ianring.com/musictheory/scales/
I appreciate that your current project doesn't need this, but if someone wanted to somehow create a MIDI file that included the information "this uses Locrian with a flat second", the current SMF spec is inadequate.
> There's nothing wrong with the idea of gameifying piano, but it's a really hard problem to solve well.
Agreed. So, my previous 'side project' I babysat for about two decades, I'm in this for the long haul and fully intend for it to be 'best of breed' and standard setting. Which is one of the reasons why we are recruiting a piano teacher right now.
> The usual result with piano teaching apps of all kinds - as someone else commented below - is kids get up to Grade 1 (maybe 2) and then hit a brick wall.
But that's already better than nothing at all, and if we can find a way around that brick wall then I see no reason why it could not go much further than that.
I don't think a piece of software can be a substitute for a teacher, but I do know that if I'm practicing I'd rather do that by myself than to have a teacher look over my shoulder for hours on end.
> The reality is that playing any instrument with non-trivial competence is really fucking hard. It takes everyone a ridiculous amount of time and effort to get good at it - even those with talent.
That we fully agree on. But that doesn't mean people can't get from zero to fifty before deciding that they need better tools and better teachers. There is room for a multitude of tracks some of which will lead to success. The main objectives for me are: to help beginners achieve a level of immediate success that it increases their interest, to ensure that they can get to some level of proficiency before shelling out a lot of money, to overcome the usual frustration that causes people to abandon their chosen instrument, to ensure that they don't learn bad habits.
> So there's a trade off between trying to offer easy but limited success with some entertainment value, and going the whole way and doing it properly - which all but the most musical kids get very bored by, because it's such hard work.
This is also very true. But it may be possible to chart a path that includes both of those in one offering, I'm not sure that we are on that path right now but it definitely is the intention.
Think of this as the beginnings of an engine, which can be used in many different ways. I now first want to get that engine perfected and then to create the rest of the application around it.
So far, especially the 'auto' mode which creates small lessons based on your performance so far has been a game changer for me and for the rest of the people that have been playing around with it, it is conceptually simple, a bit tricky to implement but now that it is there I wonder how I did without it before. The ROI of practice time invested in a piece versus my ability to play it has literally jumped up because of that.
I wonder whether that's actually true. Or maybe it depends on what level you're trying to achieve.
If you want to become a good piano player, the feedback mechanism that corrects your playing should actually come from listening yourself. A large part of the challenge in becoming a good player is figuring out what to listen for -- this is also why you need a teacher, because you simply do not hear your mistakes yourself in the beginning.
Of course here I'm mostly referring to musicality, and not just whether you're hitting the correct notes. When learning the piano one usually starts with the latter, but I wonder if you make piano more game-like it causes you to underdevelop your own feedback and reward systems, leading to more problems at a later stage.
I think the software actually helps you to spot subtle errors quite quickly, but you'd have to try it to see for yourself.
I agree, but tools like this or Synthesia can potentially help you practice the piano, which is what you should be spending most of your time on.
How many kids or adults who start with traditional piano lessons will actually make it past this level? I'm sure that it is the case that most people will eventually plateau using an app, but I doubt that a control group of people taking lessons would perform much better, and apps at least solve a problem of access.
The vast majority of children and virtually all adults starting piano will never achieve a high level of musicianship, but their lives may well still be enriched by the musical knowledge they gain.
It's true, getting very good is a great deal of work. But, you don't have to be a virtuoso to get some benefits. An app won't teach you to play well enough to give a concert at Carnegie Hall. But, statistically speaking, you aren't going to play Carnegie Hall at all.
I think the value is in option #1.
My son is 5 years old and just started grade 1; his best friend at school has two parents who are quite musical, and so he's been taking violin lessons. Thanks to COVID, he was stuck in the house, bored, all summer, and decided to spend basically the whole time practicing (mind you, the kid is barely 6) and now he's way beyond the other kids his age and is extremely proud of what he's accomplished.
Conversely, my son hates fine motor skill activities; he's not good at them, so he doesn't want to do them, so he doesn't get good at them. We couldn't convince him to practice drawing, writing his letters, anything; he didn't want anything to do with it. Until, that is, we told him (not entirely falsely) that improving his ability to use a pen, pencil, marker, etc. with precision would also help his ability to use a Switch joy-con thumbstick with more precision.
Last year, in kindergarden, he would get extremely (disruptively) upset when he even thought it would soon be time to do art, drawing, letters, whatever. Now he starts every morning in class by drawing. There are other things he could be doing, apparently, but he draws, every morning, because the idea of getting better at video games got him over his initial resistance to try something new that he wasn't good at and to improve at it.
I doubt I could ever get him to really practice piano, or to even get him interested in it, but if I put something in front of him that showed him how to play the Super Mario Bros theme I think he'd be pretty excited about it. I think I'll try this afternoon.
Sometimes it's not about getting great; sometimes it's not even about getting good. Sometimes it's just about starting in the first place.
My motor skills are _fine_, and I had a lot of issues with that throughout schooling (especially cursive, grrr, but drawing/art was also hard), but I played a string instrument (bass, and guitar), and didn't feel the same kind of frustration with that. I mean, chords are hard and kind of frustrating, but I think that's true for everyone. Piano and fretted guitars have the bonus that you can't be off pitch, as long as you hit the right one. Moving a pencil on a page is a lot different, you have to push down enough, but not too much, while making shapes in the right sequence, and you have to fit in the lines, etc, etc. UGH. Hitting the keys with the right strength is important for piano, but at a beginner level, just doing it not too hard is enough.
That seems like a splendid idea. High-res cameras are cheap, and computer vision is all the rage lately.
too often lessons are sadistic chores
ps: I also beg to differ about finger pattern, I've seen some seasoned piano players doing just whatever worked, the kinetics of playing allow for more than one pattern most of the time
If you're happy hitting grade 1 and hitting that brick wall just to have some fun, the apps do have their place, because these are people who wouldn't otherwise shell out for a teacher. It may even help people gauge whether or not they like an instrument before they move forward or change instruments.
I do 100% agree with you though that for anyone who wants to be even remotely serious, the cost of a teacher is a very good investment.
I saw a video recently [1] which claims to use AI to predict finger technique for an arbitrary MIDI file. This was always an issue for me in using Synthesia [2] - a piece of software that 'gamifies' leaning a piece. Synthesia is capable of displaying fingering information, but only if someone has manually input it - and without the fingering it's almost impossible for a beginner to learn difficult passages.
If MIDI files contained accurate fingering (by which I mean which finger of which hand...although maybe there's more to it?), and then could be input into a game - is this useful education tool?
[1] https://twitter.com/fayezsalka/status/1314613736511016961?s=... [2] https://synthesiagame.com/
Burn, but understand the sentiment :)
@jacquesm if you're looking for someone to chat about this kind of stuff with - I would love to sync up!
--Shameless Plug--
I've been working on a passion project for over 2 years now called Piano Gym! https://pianogym.com
The idea is very much the same - in that a USB midi piano (in chrome for now) can be connected to the computer and used to play sheet music in contenxt.
HOWEVER, Piano Gym's goal is much larger than Piano, and much larger than sheet music, despite that being the current state of the product.
Piano Gym's focus is to apply spaced repetition and flash card based review systems to get people playing Piano and learning music theory in context, while ALSO managing review systems and check points. Eventually we want to do this for all instruments too!
We want you in the gym to do your reps!
Additionally anyone can create content there. Whether you're a learner or a teacher you can make your own schools, courses, and lessons that others can enroll in to learn from!
It's incredibly exciting and super cool to see other people trying to solve these problems.
If anyone is interested please come and join us at the gym - Piano Gym!
https://github.com/apankrat/note-detector
https://www.pianostreet.com/smf/index.php?topic=12590.0
https://m.youtube.com/watch?feature=youtu.be&v=k50uk8elR0M
http://pianobooster.sourceforge.net/
https://musescore.com/hub/piano