Hacker News Reader

Comment by johnramsden | Hacker News Reader

What I would like to see you would be an open source speech recognition system that is easy to use, and works. There is really no match for the proprietary solutions in the open source world right now, and that is disappointing.

Not to say that there aren't open source speech recognition systems, they just aren't completely usable in the way proprietary solutions are. A lot of research goes into open source speech recognition, where they are lacking is in the datasets and user experience.

Hopefully, Mozilla's Common Voice project https://voice.mozilla.org/ will be successful in producing an open dataset that everyone has access to and will spur on innovation.

View on HN

I agree - I don't care how good Google gets it, this is an unsolved problem until I can do it with open source tools operating without an internet connection.

Google has basically invented a special processor with a very, very, VERY weird architecture for these sorts of tasks: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...

I don't think this level of computational power can be achieved on a modern CPU, or even a GPU! But GPUs are probably the closest analog to Google's absurdly parallel architecture.

To get a GPU working at maximum performance, you either have to go OpenCL2.0 or CUDA. Compared to OpenCL1.2, OpenCL 2.0 has a better atomics model, dynamic parallelism (kernels that can launch kernels), shared memory, and tons of other features.

NVidia of course supports those features in CUDA, but NVidia's OpenCL support is stuck at 1.2. So in effect, CUDA and OpenCL are in competition with each other.

Anyway, that's the current layout of the hardware that's available to consumers. I think its reasonable to expect a graphics card in a modern machine, even Intel's weak integrated-GPUs have a parallel-computing advantage over a CPU.

So for high-parallelism tasks like audio analysis or image analysis, it only makes sense to target GPUs today.

The TPU architecture isn't that weird...it's basically a hardware implementation of matrix multiplication. It also isn't a silver bullet for ASR, where neural networks are usually only used for a part of the recognition process.

Neural networks are used for nearly all of ASR now. Last I heard only the spectral components were still calculated not using a neural net and the text-to-speech is now entirely neural network (i.e. you feed text in and get audio samples out). I'd be surprised if they don't do that for ASR too soon if they haven't already.

Although some models are end-to-end neural nets, most of the ones in production (and all of the ones that get state of the art results) only use a neural net for one part of the process. Lots of people are as surprised as you, but that's the way it is.

Edit: I should say that in state of the art results there tend to be multiple components, including multiple neural nets and the tricky "decode graph" that gok and I are talking about. These are trained separately then get stuck together, as opposed to being trained in an end-to-end fashion.

Separating acoustic model and decoding graph search makes sense since you would need a huge amount of (correctly!) transcribed speech for training. See, for example, this paper by Google [1], where they used 125,000 hours (after filtering out the badly transcribed ones from the original 500,000 hours of transcribed speech) for training an end-to-end acoustic-to-word model. Good "old-school" DNN acoustic models can already be trained with orders of magnitude less training data (hundreds to thousands of hours).

[1] https://arxiv.org/abs/1610.09975

Yes, exactly. I do wonder whether a similarly good end-to-end system could be trained by constraining the alignments as I've seen done in some papers.

AFAIK state of the art models are hybrid of HMM/GMM and CNN for phoneme classification. There are exotic CTC/RNN based architectures for end-to-end recognition but they aren't state of the art.

> The TPU architecture isn't that weird

Its "weird" in the ways that matter: there's no commodity hardware in existence that replicates what a TPU does. The only place to get TPUs is through Google's cloud services.

CPUs are basically Von Neumann Architecture. GPUs (NVidia and AMD) are basically SIMD / SIMT systems.

Google's TPU is just something dramatically different, optimized yes for Matrix Multiplication, but its not something you can buy and use offline.

> but its not something you can buy and use offline.

But you will. The entire point is to put this in a phone, so you can distribute a trained neural net in a way that people can actually use without a desktop and $500-$4,000 GPU.

Is that the point? I ask, because the "weird" in the TPU is mostly its scale. Its not like you can't do matrix multiplies with the vector units on a CPU or with a GPU. Its really the scale, by that I mean its more elements than what you get with existing hardware, but its also lower precision, and appears less flexible, and is bolted to a heavyweight memory subsystem.

So, in that regard its no more "weird" than other common accelerator/coprocessors for things like compression.

So, in the end, what would show up in a phone doesn't really look anything like a TPU. I would maybe expect a lightweight piece of matrix acceleration hardware, which due to power constraints isn't going to be able to match what a "desktop" level FPGA or GPU is capable of much less a full blown TPU.

> But you will. The entire point is to put this in a phone, so you can distribute a trained neural net in a way that people can actually use without a desktop and $500-$4,000 GPU.

As far as I can tell, they put a microphone on your phone and then relay your voice to Google's servers for analysis.

Or Amazon's servers, in the case of the Echo.

I don't see any near-term future where Google's TPUs become widely available for consumers: be it on a phone or desktop. And I'm not aware of any product from the major hardware manufacturers that even attempt to replicate Google's TPU architecture.

NVidia and AMD are sorta going the opposite direction: they're making their GPUs more and more flexible (which will be useful in a wider variety of problems), while Google's TPUs specialize further and further into low-precision matrix multiplications.

Neural nets take large GPUs (or TPUs) to train. Realtime inference on CPUs has been possible since forever.

Also, I just turned on airplane mode and google assistant recognized my voice.

If this were true i’d expect to see some effort to open and standardize the hardware. Otherwise what’s the point?

It's not that dissimilar in architecture or performance to the Tensor Cores in Volta, which you can buy soon.

I wasn't aware of Tensor Cores in Volta.

I'll look into them, thanks!

You're right in that TPU's allow Google to train very large datasets faster and using less power. But I think a reasonable ASR model should be trainable with GPUs alone.

The issue previously has been a lack of large enough high quality annotated datasets, and open source ASR libraries being a bit behind or not well integrated with cutting edge deep learning. I think that's changing now though. I hope it won't take too long until pre-trained, reasonable size and high accuracy TensorFlow/Kaldi models for many languages are common.

The level of the computation can be achieved just fine with a GPU or some co-processors. What the TPU excels at is performing forward inference very efficiently. So, you can't train on it, or perform arbitrary computation that well, but if you had a pre-trained neural net, you could run it very fast and with little power.

I don't think "donate your voice" will ever work. They should incorporate it in a product and add a checkbox to share your voice/statistics to improve the system.

Why don't you think it will work? Their goal is 10,000 hours of speech, which doesn't seem like an unattainable goal to me.

The dataset is only going to grow as time goes on. They just need to get their website in order (a stats page to show where they're currently at and how far they need to go would help a lot) and throw a little advertising behind it. It's just a matter of time

I recently have been trying out open source solutions for voice recognition for a personal project and you are very correct that it lags very far behind proprietary solutions. Pocketsphinx is still very limited and Kaldi takes quite a bit to setup in a usable fashion. There were a few other options I looked at that I can't think of from the top of my head but were all in similar condition.

As the article says latency is still a problem and it's a huge problem in current open source solutions, some stuff I was testing was easily 5 seconds. I know that can be improved with configuration, but when dealing with libraries of 10 words or so, that's pretty bad.

I feel like anyone who is seriously interested in this space has been scooped up by all the big companies and the open source solutions have really seemed to linger because of it. It's one of the first areas I've seen where open source alternatives are really behind the proprietary solutions. Kind of bummed me out.

Kaldi is the best, there was just Tensorflow integration added which will hopefully speed up development (though I haven't seen any pretrained models for that yet).

Here's a blog post - http://www.googblogs.com/kaldi-now-offers-tensorflow-integra...

The easiest way to deploy Kaldi is this - https://github.com/alumae/kaldi-gstreamer-server (or a docker image of the that)

Unfortunately that Tensorflow integration didn't include acoustic modeling, so you still need to use Kaldi's neural net toolkit for that.

That's the option I was looking at using, so glad to see I'm on the right track. Thanks!

I too would be interested in pointers to the leading open source options.

Just yesterday there was a Show HN built with the https://github.com/kaldi-asr/kaldi project, emscripten-ized: https://news.ycombinator.com/item?id=15534531

When I looked a while back, CMUSphinx seemed to be the most promising option but I struggled to get it installed and got distracted with real work. Some discussions online suggest it’s still fairly poor compared to the online engines.

Snips was mentioned here recently but I haven’t taken a look at it.

I had a working cmusphinx setup at one point. It was so bad that I eventually just tore it down.

It showed promise, but I don't know if any work is being done on it.

CMUSphinx is really old. Kaldi is hard to use, but it's much better.

The first time I read this I misunderstood it's meaning. I now believe the parent is discussing the differences in usability not output. In which case I completely agree.

However I will say that for my company's use case a properly configured sphinx install produces better results than kaldi. However, getting to a point where you can say that was not an easy task.

Additionally, I actually believe that for most workloads that kaldi is likely better. Not ours though.

Open source speech recognition on Github:

1. speech-to-text-wavenet (https://github.com/buriburisuri/speech-to-text-wavenet)

2. kaldi(https://github.com/kaldi-asr/kaldi)

3. Speech recognition module for Python (https://github.com/Uberi/speech_recognition)

4. DeepSpeech(https://github.com/mozilla/DeepSpeech)

5. Natural Language Processing Tasks and References(https://github.com/Kyubyong/nlp_tasks)

FYI.

Is it an issue of open source software being inadequate, or is it the lack of sufficient training data and compute power local to your home?

I’m under the impression that Google is mostly dogfooding its open source tooling for machine learning in GCP, and actually differentiates based on trained models and compute power.

A bit of both. When data doesn't exist, people aren't motivated enough to create the open source tools which would leverage it. While Librispeech is okay for academic research, it's not enough to create a good production-grade speech recognition system.

The problem wrt open source / free solutions is data. Kaldi is open source and gets state of the art results -- but the data costs a lot of money. Training the models is doable on a commodity GPU although it takes quite a while.

I didn't realize Kaldi could get state of the art results. Do you say that because you know of people doing that, or is your comment based on knowing the architecture of Kaldi?

The 8.5% in this file is what you'd compare to Microsoft and IBM's recent ~5% results.

https://github.com/kaldi-asr/kaldi/blob/master/egs/fisher_sw...

Kaldi hasn't been in first place on that dataset recently, but it was a few years ago.

On other more researchy datasets (eg. for distant speakers or languages other than English), the best system is often based on Kaldi.

One reason for the discrepancy between quoted numbers is that, if you are only after pushing that number down and not particularly interested in getting a scalable system, then you are free to run as many systems as you like in as many configurations as possible and then try to combine their outputs (ROVER etc.).

Yeah. I don't get the impression that the Kaldi core team has been trying very hard recently to get SOTA on eval2k/switchboard. This number uses one acoustic model with a trigram LM decode + fourgram rescoring -- there isn't even a neural net language model in there. If I remember correctly, Microsoft's first "human parity" result used something like three acoustic models and at least four types of language models. This Kaldi model is competitive with the best single acoustic model Microsoft used.

Fully agree. I think their work on training data augmentation (e.g., their ICASSP paper, http://danielpovey.com/files/2017_icassp_reverberation.pdf, or the ASPiRE model before) has a bigger impact on the practical usefulness of ASR than getting an X% relative improvement over the previous SOTA on the eval2000 set.

Mozilla may not be able to make much headway until these patents expire:

https://www.quora.com/Is-anyone-working-on-an-open-source-ve...

Apparently, the business model for voice services in the past has been to snap up as many broad patents as possible to keep competitors at bay. I read an interview with a google engineer a couple years back claiming the same thing. They have to carefully work around a patent minefield with their own services and how these patents are holding back better voice search on mobile technology.

You are correct that the problem is data. Kaldi is hard to use, but making it easier to use isn't as hard as getting good training data. Mozilla's project is a good start for some purposes. One flaw with it is that they're having people read sentences. When people read, they tend to speak more clearly than when they're figuring out what to say on the fly. This means models trained with Mozilla's data will tend to need people to speak extra clearly than eg. a model trained on conversational data.

(edits: spelling/grammar)

Oh it's not "hard" to get training data, you just need loads of money to buy the existing datasets.

Hah sure. By "hard," I meant that it's the largest hurdle. And probably even the common research datasets aren't enough to give you results competitive with Google, etc. AFAIK Google uses its own hand-transcribed data.

or have more effective ways to collect tons of open source speech data. The Mozilla Common Voice project is really cool, but they should make it way easier for people to contribute.

Like, adding a mic button for voice search next to their main search toolbar on Firefox, and then ask for permission to use that data for research.

It's only a data issue if they're trying to reproduce google voice. Personally I got much better results from dragon naturally speaking 20 years ago than I get from google voice today, the cost was that you had to train it yourself first, but the benefit was it was trained for you, not "everyone". The later is the approach I'd prefer to see mozilla/OSS take.

That's fair. I think both approaches are useful in different situations. Mozilla seems to be focussing on Siri-like use-cases as opposed to dictation. Even for dictation, for many people having to train the system themselves is more work than they're willing to do. I'm sure what you want will exist eventually :)

Speech synthesis is in a similar situation. The FOSS options that I know of are completely primitive compared to their proprietary counterparts, particularly those locked away behind the cloud. Unfortunately this area seems to receive much less attention than speech recognition (AFAIK it's a non-goal of the Mozilla project, for example).

I suspect that it’s widely recognized that getting incremental advances in speech synthesis is really hard and, unlike potentially speech recognition, it doesn’t really solve a problem that people have. For business services and consumer devices slightly better voices are valuable but, for example, the improvements in the latest Siri voice don’t make any new things possible. But it’s part of the fit and finish of an expensive phone.

I'm the cofounder of Snips.ai and we are building a 100% on-device Voice AI platform, which we want to open-source over time

You can build your voice assistants and run them for free on a Raspberry Pi 3, or Android

Reply on news.ycombinator.com