Mozilla partners with Nvidia to democratize and diversify voice technology | Hacker News Reader

Mozilla partners with Nvidia to democratize and diversify voice technology

blog.mozilla.org

44 pointsVinnl14 comments

Mozilla TTS is a great open source speech generator that uses state of the art ML models. https://github.com/mozilla/TTS

If you need good TTS, this is it. It sounds as good as Google, Apple, or Amazon products, if you have a good data set. The pre trained models are decent enough for real use cases.

Datasets are the problem. Quality data is expensive. It takes about 24 hours of aligned and annotated quality voice samples to make a model.

The most common "open" dataset is LJSpeech. It's kinda noisy and recorded with a bad mic. And that greatly degrades quality of generated audio. It's still miles better than anything 5+ years ago, but has a metallic quality.

If anyone is interested, you can record your own voice to make a model for Mozilla TTS. What isn't talked about, is that modern ML voice models aren't synthetic voices, they're audio deepfakes. Clones. We're talking to the point your friends couldn't tell if it was you or the model on the phone.

Some people might find that scary but I think it's incredibly cool. If you build a great quality model for Mozilla TTS your voice can outlive you.

Is there a guide you could recommend on building your own Mozilla TTS model? Or is the repo you linked to sufficient?

It's barely sufficient. Much of the docs is in various pull requests from others adding models. In another reply a user posted a better maintained fork repo

Thank you!

I think the creator of the project has moved on and it is not maintained anymore. But they have https://github.com/coqui-ai/TTS

Quite sad, glad someone picked up the torch

I hope all this data helps us get better free text-to-speech libraries even for English. There's good quality speech synthesis built into OSs like Windows and MacOS and online services from Google, but if you want to include a library in your own local software, the existing open source options are all equally Steven Hawking sounding.

Why include it in your app though? That's inefficient and inconsistent. If your OS has speech support, using that is going to be the best experience for the user. We definitely do need a good FOSS speech library, but it should only be used in a backend for speech-dispatcher and backend/headless apps, not anything user-facing. The OS exist for a reason.

Security.

Privacy.

From your OS? If your can't trust your OS, you have bigger problems.

Linux doesn't. It'd be nice if all OS's did but until then, you have to do it in your own app. Why only as a backend? Why not just compiled into applications that use it like any other library?

There was a related announcement 1 month ago:

https://old.reddit.com/r/firefox/comments/m18rbg/donate_your...

It would be great if we start democratization with some fundamental needs first: the drivers.

Hopefully they'll be able to give DeepSpeech some more love as well.

Reply on news.ycombinator.com