Hacker News Reader

Comment by asaddhamani | Hacker News Reader

From a quick try results aren't good. Sounds bland, and the text I type isn't exactly equal to the text that is spoken. Didn't try with voice cloning though.

Why is good TTS so expensive and why are there no good open source options? Is it just from the need for high quality training data? I don't imagine these models are more expensive to run compared to SOTA LLMs, yet they cost so much more.

View on HN

From what I'm seeing, most of the open source TTS models are trained on the same few voices, mostly in 16Khz, mostly from Librivox books I think.

Eleven Labs is most likely trained on stolen audiobooks, they've published a few Youtube videos in Polish, now taken down, of AI renditions of famous Polish audiobook narrators. This was all before they became popular, and before their voice cloning models were publicly available I think.

> mostly from Librivox books

That probably explains a lot. I've tried listening to some of those audiobooks - very hit and miss, mostly miss. Definitely amateur hour and mostly bad quality.

I had pretty good results with coqui-tts and a VITS model, I trained myself with an open dataset and later with one I extracted from audiobooks / epub and therefore can't publish (german)

The dataset and video tutorials are all available and linked on (also english):

https://www.thorsten-voice.de/en/motivation-vision/

Thanks for mentioning my Thorsten-Voice project, dear sandreas :)

You're very welcome.

a few weeks ago i used piper to create an acceptable translation of a book. i didn't listen to it all, but the result sounded better than anything i was able to listen to before. good enough to listen to a book if a human read one is not available. just a few years ago, this was not the case.

in other words, while FOSS TTS lags behind commercial options, it does get better and i expect within a few years it will produce results that are at least as good as the commercial options today if not fully caught up.

Piper seems roughly equivalent to old-school TTS outputs that sound flat, jumpy with the concatenative approach. Listen to this first example I tried:

https://rhasspy.github.io/piper-samples/samples/en/en_GB/ala...

Of all the TTS APIs I have tried, I like OpenAI voices the best. Haven't considered things like elevenlabs because I find them ridiculously expensive.

I love voice to voice interfaces, but only when they sound natural to my ears, and the current pricing for good ones is prohibitive for a huge number of use cases.

well, i was comparing it to the free tools available a few years ago, and against that, this example is a markable improvement. it's the first that i could actually bear to listen to over a longer period of time. i expect just another few years and this will actually be good.

There are a lot of options. StyleTTS2 is pretty good, XTTSv2 is pretty good, the new E2 TTS and F5 TTS also seem decent.

Commercially available high quality training dataset is the key. Open search libraries don't get the luxury of working with voice actors to record voices.

Would it be hard to create such a training dataset? Seems like you’d just need a lot of people to say a bunch of stuff for you?

needs a crowdsourced model

Ideally, Mozilla would step up here given their mission statement, but they won't, probably because their CEO needs another bonus.

Yeah there's no chance Mozilla would do anything like this:

https://commonvoice.mozilla.org/

That's the first thing I thought of! I wonder how used these are. Are there any sources or data points indicating that this commonvoice data is being used, and if so, where/how? I think I may have contributed to this a few times back years ago. Nice to see it's still going, would be better to know it's being used.

It was used quite a bit of speech to text - but tts it’s not that great.

It costs a million dollar a year to host 32k hours of audio?

Have you tried VoiceCraft?

Yeah all these seem hyper focused on "voice cloning" so on replicate VoiceCraft doesn't even let you try normal TTS unless you provide a reference voice so I noped out.

Reply on news.ycombinator.com