Show HN: Voice Clones for Creators | Hacker News Reader

Show HN: Voice Clones for Creators

coqui.ai

71 points_josh_meyer_63 comments

Remove step one from the video. Sign up is assumed, and it’s boring, and I dont need to be told how to put my email in the box that says email. Or organisation name in the…

The point of the video is to show me the value. Show me what I can create. Within the first 10 seconds I should say “wow, I want this”.

Start with a result, and then go through the key! Portions of the process, if you feel that’s the value.

There’s no cloned voice in the whole video…

There's three voice demos on the front page.

Yes, but I agree wholeheartedly with the recommendation that you are responding to. I clicked on the first big “play” button I saw, thinking it would be a quick demo. I then started skipping through the video to find the meat. In particular, I expected to hear someone’s cloned voice and say, “wow”. When I realized it was primarily some stock footage and some silent recording of the UI, I moved on, and THEN realized that there were demo voices.

yours is a solid point -- would be nice to have that "wow factor" right off-the-bat

They are not even visible on the first visible page on a mobile device.

Learn from this….every second you have someone’s attention is precious.

I would just scrap the video entirely.

Are there are citations/verification for the "used by" logos?

Basically they have every FAANG there, they're claiming that these companies use this, yet certainly at least for Google they have their own very active research and products going on in this area (DeepMind, wavenet etc) so it feels fairly implausible to me that they'd not be using their own tech that they are actively researching themselves, and instead use this?

Feels a bit fraudulent to splash these names and logos around without any proof?

I think sometimes folks put these on there for things like, an issue posted on github by a user whos email has a certain domain, or other very flimsy acknowledgements.

Maybe overly cynical, but I usually assume it's "someone with an email address from that company signed up for a free account because they wanted to spend 30 seconds trying out the service"

I dont want us to be right, but damn. It could be what is happening in a lot of cases.

Looks like they've got some decently popular open source repos - https://github.com/coqui-ai/TTS for example - so I wouldn't be surprised at all if Google and others used that, even if only for their own research and comparison. That's my charitable interpretation anyway.

Voice Clones for:

-> Video Games https://coqui.ai/video-games -> Dubbing https://coqui.ai/dubbing -> Post-production https://coqui.ai/post-production

Free access

The video games one is funny. The hero text is "Emotion on Call" and the video title is "hearing is Believing". The video is narrated by the most passionless voice imaginable. It barely has question inflections.

I've wondered if this tech could be useful for video games... but it's going to take more than just recording one "voice" sample and typing some text. At the very least you're going to need some formal notifications of tone, because there's a world of difference between "[sarcastic] Yeah, really." and "[emphatic] Yeah, really." and at times its a super-AI-complete problem to know exactly what the author intended without annotations. (That is, even humans would get it wrong sometimes.)

Unfortunately for the company, that video is indeed a great demo of the fact you can't just shove their technology at some text and "solve" the problem.

However, for the first time, I feel like a half-decent solution to this problem is on the horizon, rather than absurdly far away.

Hasn't been linked yet so: https://github.com/coqui-ai/TTS Derived from Mozilla's TTS project. Big fan.

thanks for linking -- the core project is indeed open source! the founding team all worked at Mozilla on TTS and DeepSpeech, then we spun out to create Coqui <3

I've tried a few of these over the past few years. This is by far the most impressed I've been by the generated result. I'm actually an educator and have used these services in the past with students as an entry point into a discussion about technology, AI, and media ethics. Is there anyone I can reach out to at Coqui for a possible collaboration?

of course! we've got an open ethics discussion going here: https://github.com/coqui-ai/TTS/discussions/1036

and our chatrooms are probably the best/fastest way to get ahold of us: gitter.im/coqui-ai/TTS

Same here. Not only the result is the best so far and instantly useable, the website is great to use and the fastest.

The voices do lose a bit of character and accent, though. On the other hand that makes my own voice much more useable. That's probably for the better, and good for video narration, audiobooks and marketing.

But I'm curious about the Video Game offering [1], where it says "You control the enunciation and emotion; the pitch and prosody; rate, duration, and contour". Is that planned for the future?

[1] https://coqui.ai/video-games

more fine-tuned control over enunciation/emotion and the like is very much in the works... stay tuned :)

That's pretty cool! Easy and fast to use, with reasonably good speaker reproduction.

Right now the free version only allows a sentence or two of output. Are you moving towards a paid plan with longer outputs?

And the training right now is just a paragraph of text. Does it get better after additional training?

For this zero-shot approach, there's actually no "fine-tuning", and from what we can tell ~30 seconds is optimal (the text isn't actually even used... you can say anything!)

For longer outputs and legit fine-tuning, that could become an offering :D

Is it possible to do the same thing (zero-shot on an arbitrary input voice) using one of the models on your Github?

Great to see how things have progressed Josh. My sample was good - not likely to make anyone think it was me (it gave me a distinctly American twist), but it's an impressive zero shot result.

Btw the website looks slick too - works well for me on Android 12 / Chrome

Thanks, Neil :D The current model definitely does an "implicit" accent conversion to American English, but in the near future we'll have something that keeps accents on separate sides of the pond, so to speak :)

This is great! I see you've open-sourced some stuff, is there something I can use to train and generate these voices on my computer? I'd love a short tutorial somewhere of "you have an MP3 of your voice, here's how to generate more".

delicious documentation for your reading pleasure:)

https://tts.readthedocs.io

Excellent, thank you!

Seems to have turned me into an American...

I would definitely be willing to use a service like this. I produce a lot of YouTube content and recording and editing good quality audio takes a lot of effort.

"JSON.parse: unexpected character at line 2 column 1 of the JSON data" on firefox(bad request to graphql)

Yea, 400 error on their graphql endpoint

Is there an API for this? I actually have a programmatic use case I'd love to look into using this for :)

currently it's only user-facing, but still early days:)

Great to see other Germans working on ASR and TTS :)

I'm looking forward to having an offline and private voice assistant one day...

In case you're not a member yet, take a look at LEAM.AI for potential partners and/or government funding.

When Descript[^0] came out with automatic overdubbing, it was absolute magic and it stayed magic for a long time. Happy to see that there is now an open-source alternative, although I'm not sure how easy their models are to use.

Does anybody know how they prevent you from creating a voice that isn't yours? I expect the text they make you read to be random, but if you have a large enough corpus of the person you want to imitate you could just piece it together.

[^0]: https://www.descript.com

I get an error at /voice/create: Unexpected token < in JSON at position 1

Google Chrome 102.0.5005.61 on MacOS Big Sur

still getting the error? I can't replicate on MacOS Monterey

me too on an iPad using Firefox

Well, I'd like the voices of David Attenborough and Christopher Plummer for audio books please...

https://github.com/BenAAndrew/Voice-Cloning-App

Also, this timely announcement that Spotify has acquired Sonantic: https://techcrunch.com/2022/06/13/spotify-is-acquiring-sonan...

Don't forget Morgan freeman

Indeed! I'd like to add Lance Henriksen, too.

I'd like to be able to generate a voice like HAL, from "2001, A Space Odyssey".

why don't you try :D ? Make a clone where you're talking like Hal... see how it does ;)

15.ai has it

A future refund scenario might be asking for David Attenborough's voice and getting Glibert Gottfried's instead.

Using the Brave browser, the examples don't work on your home page.

Thanks for reporting, will take a look.

Product Hunt --> https://www.producthunt.com/posts/coqui

walk-though video --> https://www.youtube.com/watch?v=ri1U-bJ-6vc&t=4s

You are really missing an opportunity by not including the actual audio recorded and the sample played back.

the voices are demoed for [videogames](https://www.youtube.com/watch?v=x8tEdwll_CY&t=4s), [dubbing](https://www.youtube.com/watch?v=TBjdUY3_ccQ&t=1s), and [post-production](https://www.youtube.com/watch?v=ykoBFrb9itY)... but I see your point - thanks!

video game demo --> https://www.youtube.com/watch?v=x8tEdwll_CY

On mobile:

The page is too wide.

The demos don't play.

thanks for flagging! what OS are you on? iOS seems working OK atm

Under Firefox on Android the demos play, but I actually need to click left of the play button for them to start. Looks like the radius of the time slider is too wide and overlaps the play button area.

Perhaps that's what the parent comment is seeing?

thanks for flagging!

had the same problem on Safari iOS. Worked only after I accepted the cookie consent form.

This looks like a repeat from 6 months ago:

Show HN: Clone your voice and speak a foreign language - https://news.ycombinator.com/item?id=29786132 - Jan 2022 (112 comments)

Usually the cutoff for reposts is a year (see https://news.ycombinator.com/newsfaq.html), unless the project has had a major update since then (see https://news.ycombinator.com/showhn.html). In the latter case, the post should come with an explanation of what's new since the previous thread.

same company, yes, but major updates since our demo:

1) Quality: the underlying model is much, much better, and so is the quality of the Voice Clone. 2) Productization: previously we just had a stand-alone demo, now we've launched the product (user accounts, multiple voices, etc.)

Ok, that's probably fine.

Reply on news.ycombinator.com