Comment by yorwba | Hacker News Reader

Comment by yorwba | Hacker News Reader

Parent Full thread

Have you considered adapting your app for deaf people?

A friend of mine developed a pronunciation practice app for deaf speakers of Korean as a student, which won them an innovation award. But according to my friend, the pronunciation score (which was based on comparing spectrograms or something) didn't work all that well, and the award was mostly due to the non-technical team members making up a heart-warming story to promote their project.

I assume your feedback system is more advanced (can you share some details on how you determine what recommendations to make?) and would also work for deaf speakers. But you can't really assume familiarity with another spoken language, so you'd probably need explanations specifically tailored to deaf people.

The problem with adapting this to other forms of atypical speech is that their recommendation system likely relies on a catalog the phonemes L2 speakers have issues with (the example most people know is the Japanese "L" "R" swap) so it's much easier to create courses with specific focuses and solutions.

If Google's Project Euphonia [0] is actually still ongoing and they release their dataset/methodology of training models with that sparse dataset I can see your idea as approachable; even accented speech is a tough problem to work on considering how many variants exist worldwide (but their approach looks good!).

[0] https://sites.research.google/euphonia/about/

That's an interesting use case that we could potentially explore down the line, thanks for sharing!

Right now, we're focusing on non-native English speakers, as it's a problem that we personally understand best -- as well as a very large market.

I just want to note that for many deaf people (including practically all people that were born deaf) sign language is their first language, and thus, technically, English isn't their native language.

edit: I'm also curious to know more about what kinds of analysis/algorithms you do to detect the accent, if you can share a little bit. Can your app distinguish between different American accents? Or could it hear the user talking in African-American English and assume it is "incorrect"?

Reply on news.ycombinator.com