Hacker News Reader

You seem to be using a slightly tweaked CTC-based architecture built in tensorflow (possibly with Baidu's warp-ctc) but marketing it as some super-secret technology you invented in-house. I don't see any performance benchmarks or WER results we can compare with other APIs, but the pricing is the same. Surely character-based approach lets you add new words without pronunciations, but that process is not as flawless as you make it seem, especially when you lack language model data for new words. Now I'm still a bit confused why somebody would use AssemblyAI over other APIs given the same price. And FYI you are not using Kaldi / Sphinx because the guys behind them did not endorse CTC and are purposefully avoiding putting it in there, though for example Kaldi's chain models are also sequence based. There was also Eesen that tried to implement CTC on top of Kaldi. Sorry if this came off too harsh, but I am a little suspicious about the novelty of the approach here.

I remember 10 years ago Nuance used legal threats to eliminate competition in this field, to the extent that greatly discouraged any startup speech recognition companies.

Google was able to get around it, just because they became heavier..

Did this significantly change since then?

> We've developed our own deep neural network speech recognition architecture, and aren't using any open source speech frameworks like Kaldi or Sphinx (just Tensorflow). Because of this, we're able to run things more affordably and pass those savings on to developers.

Kaldi and Sphinx are far more efficient than any tensorflow transcription model I've ever seen.

I assume this is an oversight ?

We can get pretty good throughput with our setup. But the main thing about our architecture versus others is that it makes the automatic customization we offer possible. For example it takes under 2 minutes to customize the API to be able to recognize ~10,000 custom words/phrases for whatever you're building.

Your pricing page contains no pricing information.

Don't you just hate that?

Haha totally get this. We wanted to give info about how we bill (ie, per second and we don't charge for customization) but we haven't publicly disclosed our rate yet. It's pretty low (fraction of a cent per second).

Fair enough, thanks for providing that info.

How many fraction of a cent.

I'm sure you're aware of this, but for those who aren't, your YC buddy scaleapi.com offers human transcription for a fraction of a cent per second as well.

With scaleAPI, you also have a 1 day delay in getting your response.

Yup! Scale has a good API for human transcription. We're an order of magnitude cheaper than human transcription services though.

Been wanting something like this for years. I have a bunch of old speeches and radio shows I'd like to transcribe. They all have "terms of art", and noone at Google would tell me how to train their API to adapt to my use case. Too bad I missed this Beta; hope you allow more people in soon.

Can you clarify: does your API allow me to run the transcriber, pause it when I see an error, tell it what the corrected text is, then continue with that correction taken into account?

The way it works right now is that you would upload one of those old speeches or radio shows to the API, and then you'd get back a transcript that includes all the "terms of art" you customized the API to be able to recognize. Right now there's no feedback loop for you to tell the API where it was wrong, but that's something we have in mind to build! We do run QA across our entire API usage though, and from that are able to improve recognition accuracy over time. We also release updates to the models that power the API every few weeks which improves recognition accuracy too.

And then we do provide confidence scores, so we can at least give you some indication when we're not confident in the automatic transcript we're returning to you.

If you want to try out the API, you can email beta@assemblyai.com and I will look for your mail!

Thank you, I will. I also wonder, does your API take context into account? For instance, depending on WHO is talking, and WHAT they've recently talked about in the recording, one transcription may be preferable to another. People often recognize other people by the types of things they talk about, characteristic phrases, etc.

Email sent. If the spam filters blocked it and you didn't see it, post here please.

Small issue I notice that the email links on the pricing page: they're swapped, with "Basic" having an "Enterprise Plan" subject line and vice versa

Good catch! Thanks!

No problem! Can't wait to try out the beta, especially the streaming audio transcription feature!

Any WER benchmarks for TED, Librisvox, etc?

Right now we only test against a few internal test sets. On some clean speech test sets we're at ~8% WER. That's a good idea to test against some open source datasets for reporting accuracy metrics!

Shouldn't that be before you release MVP?

Maybe a silly question but could you use this to recognize phrases or words in a language other than English?

We haven't actually tried that yet. I imagine if you customized your model with words from another language and then pronounced them with an english accent the API might be able to recognize them ok. Would be a fun experiment to try at least!

If I understand correctly, "customizing the model" essentially adds new words to the vocabulary and adjusts the language model to change the probability of some phrases, but does not require any information about pronunciation, let alone audio samples.

But isn't having just the English text really error prone, especially when you are dealing with terms of art and proper names, that might even have roots in foreign languages? E.g. some people pronounce SQL as "sequel", and the English pronunciation of French words varies between "French pronunciation with English accent" and "French orthography interpreted as English orthography". (I'm guessing your model would tend towards the latter?)

So what I'm interested in is whether you have encountered examples of this during your testing, and whether you have some way to work around it (I would try phonemic transcriptions in addition to English); or whether this is not relevant for the use-cases you are trying to cover and the convenience of just using English text trumps the accuracy loss due to just using English text.

You can create phonemic transcriptions as a back-off for unknown words (at least in WFST based setups), but with things like "sequel" this won't help much.

AssemblyAI is apparently using their own TensorFlow implementation, not weighted finite-state transducers like e.g. Kaldi.

Speaking about WFSTs, why wouldn't it work for "sequel"? I have only done the "Kaldi for Dummies" tutorial (i.e. digit recognition), but from what I understand, you could add an utterance "s iy k w eh l"/"SQL" and add phrases like "SQL query" to the corpus and this would make it more likely than "sequel query".

Hey! Great question. Our system is actually able to handle transcribing "sequel" as "SQL" automatically if you were to "customize the model" for phrases like "what was my latest SQL query". It can also get words like "colonel" pronounced "kernel". In both cases, without needing the explicit pronunciation of the word. We have some customers who've uploaded thousands of proper names, for example, and we're able to transcribe all of them without needing the explicit pronunciation. This is possible because our ASR implementation is pretty different than traditional setups like Kaldi. You're right that there are some edge cases, especially with foreign words, but we're working hard on smoothing those out.

Sounds amazing! Now I'm really interested how your setup can do that. Will you publish anything about it, or is this the kind of secret sauce you'd rather keep secret?

Is your product compatible with medical privacy law? Could it be made to be compatible with such law?

This is something we are looking into!

Can you separate multiple speakers in audio when you do the transcription?

Thanks for the question! If you have a stereo file, from a phone call for example, we can do separate transcripts for each channel...but we haven't launched any algorithms to auto-split speakers into the API yet! Definitely something we want to offer in the future though.

Just FYI, the cta buttons near bottom overlap on mobile

Thanks for the heads up on this!

Similarly menu items in the navbar become unclickable on mobile (or at the very least when emulating iphone6+ in Chrome)

Thanks for taking the time to post this! On it...

Any plans on a JavaScript SDK?

Yup! This is in the works and should be released soon!

Your pricing seems on par with google and ibm

Great work guys. Was excited to see AssemblyAI is free for open-source projects. Looking forward to see big relevant projects hop on the train.

Launch HN: AssemblyAI (YC S17) – API for customizable speech recognition