Getting Deep Speech to Work in Mandarin | Hacker News Reader

Getting Deep Speech to Work in Mandarin

svail.github.io

58 pointskornish18 comments

The amazing part is that their system seems to be adaptable to any language with a minimum of human effort.

   > One of the reasons deep learning has been so valuable is that it has converted 
   > researcher time spent on hand engineering features to computer time spent on 
   > training networks.   
 
   [...]

   > We can now train a model on  10,000 hours of speech in around 100 hours on a 
   > single 8 GPU node. That much data seems to be sufficient to push the state of the 
   > art on other languages. There are currently about 13 languages with more than one 
   > hundred million speakers. Therefore we could produce a near state-of-the-art 
   > speech recognition system for every language with greater than one hundred 
   > million users in about 60 days on a single node.

The effort definitely not minimal. You have to collect 10000 transcribed hours first. This is not easy in many languages.

I'm surprised it doesn't use Character Aware Neural Language Models (CNN -> LSTM RNN) but instead a layered RNN. Interesting!

Facebook disallows some images, based on the personal standards of whoever happens to be in charge of image disallowing that day. Google controls what you see based on your own past, limiting your exposure to opinions you might not like. Companies comply with oppressive government requests for control and surveillance.

If we surrender our ability to communicate with people speaking in foreign languages in this fashion, we will literally become unable to talk about things that we "shouldn't", and everything we do talk about will be on permanent record and monitored in real-time for dissent and to target adverts at us.

I keep reading about these algorithms that are "better than humans". Perfect image recognition, perfect speech recognition, parsing plain text-queries and answering questions, etc, etc. So where are the practical implementations?

All the speech recognition engines I've interacted with so far were awful. Not just bad, awful.

>Collecting such data sets could be very difficult and prohibitively expensive.

Uh, movie subtitles?

If you used Baidu's services, you might be more impressed. In any case, one barrier to rolling these things out to production is that it takes time to get them into production, and there are cost challenges: you represent too little ad revenue to justify a whole GPU devoted to you. Baidu has been experimenting with aggressive model reduction and batching up tasks and efficient GPU server design, but it's still a challenge to roll out the best deep learning results on fractions of a penny budgets.

Why does it have to be driven by ad revenue? There are plenty of people and companies who would gladly pay good bucks (relatively speaking) for a decent transcription.

I'm sure one could: https://news.ycombinator.com/item?id=11127534

But rolling out a paid service like that comes with a lot of expectations that being part of a free app doesn't.

If you use speech recognition systems on mobile (or a public web API), they are often handicapped due to space or processing constraints. Full on recognition models in clean environments (no background noise, other speech, no compression e.g. something like a landline) backed by maximum hardware are quite good for English - we have much farther to go for other languages, which is one reason Baidu's work is so interesting.

Movie subtitles have poor alignment, usually contain multiple speakers (sometimes talking at the same time) and often contain sounds or other things which are not dialogue. Cleaning this is expensive, probably much more than just getting transcriptions of single speaker samples. It corresponds much closer to "real human life" but that is not where papers are published, unfortunately.

We have thousands of hours of ebooks (librivox - librispeech is a 1K hour subset used by many) that are used in open source speech recognition systems, and Baidu has a direct line on many more hours than that.

The better than human line is (in almost every paper - Baidu is not alone in this at all!) bullshit though - while the system is quite good, they only compare to humans given a fragment of a statement without context. Humans are very, very good at inferring given context (would you recognize speech the same way at a funeral/dinner party/college lecture/hospital?), and this is where most models fall flat.

Conditional language models can help this to an extent, but human adaptation and the ability to generalize will take a while to catch up with.

> Cleaning this is expensive, probably much more than just getting transcriptions of single speaker samples

So, deep learning to clearing background noises?

> Uh, movie subtitles?

Movie subtitles are very rarely actual transcriptions of what is spoken; they are instead summaries, edited for brevity and quick comprehension.

I don't know much about what kind of corpora is required for training this kind of model, but subtitles don't seem appropriate.

Is that necessarily a bad thing? Maybe if you're transcribing text for dictation, but for many voice recognition uses, you would be happy with text conveying the semantic content of the spoken text.

This is really not my field, but my suspicion is It Just Doesn't Work Like That.

Not my field either, but I wouldn't have thought that mere word proximity would lead to something as interesting as word embeddings either, and they seem to be a general purpose "input token -> semantic meaning" mapping, so who knows? (Seriously, if someone does know, tell!)

To train a speech recogniser on subtitles, you need to be careful to allow for words missing from the transcriptions. But it has been done: goo.gl/Syx27j

In Mandarin they are 99% exact transcriptions.

Subtitle formats specify the time during which a subtitle should appear on screen; this coincides only very weakly with the time during which it's voiced (and contains no information at all at a granularity smaller than "whatever fit on screen at the time").

And for reasons that have always escaped me, subtitles frequently just don't say the same thing the soundtrack does.

It's a lot better than nothing, but subtitles make for a pretty low-quality dataset.

Indeed, subtitles often seem to be written for people who read slower than people talk.

Reply on news.ycombinator.com