Hacker News Reader

Comment by gh1 | Hacker News Reader

TL;DR A deep learning algorithm for searching words ("freedom", "Obama" etc.) in a recorded audio/video clip.

The title is clickbait, but the claim is rather interesting. It says that it has 80 % accuracy for transcribing an audio clip compared to 20 % for speech-to-text.

View on HN

Hey, I'm the blog post author (Deepgram cofounder too)! Sorry if the accuracy wasn't clear enough. The numbers quoted are for search accuracy, not for speech-to-text accuracy. It's a subtle thing but makes a lot of difference (and it is the thing that matters most when you are in trying-to-find-something mode).

If you ran phone calls or tape recorder audio through a speech-to-text engine then the word accuracy rate is like 10-50% (i.e. abysmal). When you try to search for keyphrases like "frolicking kitten" the likelihood of a text match with STT is ~20%.

If you ran that same search with Deepgram then 80% of the time you'd find what you are looking for since Deepgram doesn't have to guess at what is being said, it takes the inverse approach and matches 'how it sounds' using deep learning voodoo magic™.

Is it intentional that your website has no pricing information or specific details what one would get after signing up for an API key? Some blogposts reference pricing, but no idea if those are current and what the APIs look like. Not very inviting to play with stuff :/

I've been using Deepgram for building https://www.findlectures.com - I don't know about the marketing site but there is pricing within the app.

If you sign up it's 5 hours for free then 5$ for 6.7 hours. I figured it out just now.

So their old blogposts are inaccurate...

There are a number of papers tackling a similar task [0][1][2] for anyone who is interested. There isn't enough information to tell exactly what is going on with Deepgram, but one way to approach this would be to construct a shared embedding space for words/phrases and speech. These types of embedding spaces are powerful [3][4][5], but not magic.

Cool demo, looking forward to seeing more detail about what is going on. However I would quibble with the STT WER quoted above. Maybe in noisy environments with unknown speakers (and no voice normalization) this is accurate, but the kinds of clean speech in the demo perform really well in modern recognition engines (on benchmark data, to be fair c.f. MSR 6.3% and IBM at ~6.9%).

Most word searches over speech to text work over soft matches (or ideally beam search over most likely partial phoneme/word part matches), rather than hard matches so it seems like a bit of a straw man comparison in this case.

[0] http://research.google.com/pubs/pub42543.html

[1] https://arxiv.org/abs/1510.01032

[2] https://sigport.org/sites/default/files/gloveNNLM_kaudhkhasi...

[3] https://arxiv.org/abs/1502.03044

[4] http://www-personal.umich.edu/~reedscot/files/icml2016.pdf

[5] https://arxiv.org/abs/1411.2539

I thought the figures sound a little suspect too - if automated text-to-speech was really that bad, then presumably products like Dragon Dictate etc would never have been successful. Obviously they have the benefit of training, but still, 20% sounds ludicrously low.

20% is very low. We use machine transcription at work, and although the per-word confidence of machine algorithms varies wildly given the quality of the audio and the speaker's mannerisms. It can easily get into the 80% range with good audio, especially televised audio where people are close to microphones and there is not a lot of non-speech noise.

I had a friend working in speech recognition at Nuance for several years. I'm trying to remember what sort of accuracy he quoted to me about two years ago. It was something large like 80% or something and that getting the last stretch of accuracy was really hard with the techniques they employed back then. That figure may have been for a very specific application, which was a lot of phone menu stuff. He then moved on to Siri at Apple.

Microsoft claims 93-94 percent correct, or in other words "6.3% word error rate"

http://www.zdnet.com/article/microsofts-newest-milestone-wor...

I think you need to take into consideration that this is designed to be used on voice where the speaker is not aware that they are speaking to a computer (or that they are being recorded at all).

When people talk to Siri, Cortana, or Dragon, they take unnatural care in the clarity of their speech compared to normal talk only meant for humans. Also the speaker may not have been speaking directly into a microphone, lots of background noise, etc.

All of these factors probably combine for a much lower accuracy than what Apple, Microsoft, and Google are going to be dealing with in usual cases. Also keep in mind they all have incentive to inflate their own products accuracy score. Not that the same incentive doesn't also exist for this company/product.

Reply on news.ycombinator.com