I noticed that a very simplistic LSTM neural network model is able to learn very quickly all the rules of pronunciation ex:
- grapheme "EA" in “please” or "heat" is pronounced one way
- but grapheme "EA" in « death » or " bread" is pronounced in a different way
- but grapheme "EA" in "great", "steak", "break" is also pronounced in a different way In fact the neural network is learning the rules and able to guess the pronunciation of word it have never seen before.
This made me try to see
1- what is the minimal number of example the Model need to be trained with to learn all the rules.
2- What is the optimal sequence that rules must be learned ... This can all be discovered and measured easily and accurately by training the model with different training set. I believe this have never been done before, because experimenting with real kid is too slow.
We make the assumption that is something is easy to learn by an LSTM model it will also be easy to learn by a human. (turn out to be true)
There is a lot of design decision that need to be made about how to visually display "grapheme" and let the kid interact with them.
This is just a side project for now but I am in strong need of a UI programmer to partner with!
But in another way this could be a kind of overlay of information. Maybe it shows how words are split, maybe it's to disambiguate different pronunciations of "ea", ... I'm not sure what's the most important added information. But overlaid on the words it might be a kind of scaffolding that makes it easier for kids to successfully read words and gain the practice that makes it possible to remove that scaffolding.
There's a lot to think about there when it comes to reading (not to mention a ton of pedagogical knowledge)... I'm not sure I have the space to be a solid collaborator, but I do find this stuff interesting to talk about sometime...
Disclaimer: yes, I'm affiliated :)
I contribute to Optikey and was involved in OpenVoiceFactory in its first incarnation. Optikey is primarily QWERTY based but does supported the Communikate pagesets - more general OBF support would be a welcome PR! Coughdrop is probably a better fit for your needs, and is open source so free to self-host, though they do offer hosted plans for $.
Also what kind of art do you make? Im an artist working as a dev as well. I know we're out there but we're a bit rarer than I was expecting.
As for art I love watercolor, gouache, ink and brush. I trained professionally to be an animator so I love cartoons.
I have a 12 year old autistic son. He attends ABA therapy. I am also looking to create tools and apps that can help him communicate better. Also, he's very musical, so I am looking at better ways to help him learn piano (he loves Simply Piano but I want something that let's me define a custom lesson plan based on MIDI files of songs he loves).
I'll ask my wife who is a BCBA to share any studies related. The premise is symbols to help a kid reach from using symbols to letters/words. Right now they print a ton and velcro them to a board and communicate a complicated topic. There are apps out there but they kind of are lacking IMO and hard for a kid to figure out.
Make sure to do a Show HN when you get it far enough!!!
Have thought a lot about this space, too, and identified similar needs. Please connect via email in profile.