Hacker News Reader

Comment by bongobingo1 | Hacker News Reader

Will we see this level of voice synthesis in the public domain? Maybe I am out of touch but I found those examples very impressive - more impressive than the jobs vs rogan demo a few months back.

But I am also saddened at a future where all this is locked up in corporate hands - obviously there is money needed and (licensed) data needed too which Apple can get at.

Honestly I would rather eschew the ethics of it and just consume any and all voice data (youtube, podcasts, existing audiobooks, radio) that has transcripts available, perhaps because I assume corpos are already doing this, if it means we can have a free and open data model that people can run at home, maybe that makes me evil.

View on HN

> saddened at a future where all this is locked up in corporate hands

I would guess this rolls out from big companies first because the first version is always the most difficult. It’s only going to get easier to do and I would totally expect end-user controlled TTS systems to get better and eventually exceed the capabilities of this version from Apple. Of course Apple isn’t going to sit still, so they will continue to improve as well.

Are there examples from ten or twenty years ago of a technology that big companies had locked up that never made it out to end users? What we have might lag, but it seems like this stuff only ever gets easier to do.

Linux is a prime example of a technology that was at first far behind its proprietary counterparts but eventually dominated and nearly extinguished all non-free competitors.

For very specific uses. Linux is not a good options for general computers used by everyday people. That experience is still owned by large corporations.

There are only two cases when you can't use Linux as you daily driver:

    * You must use proprietary software not supporting Linux
    * You must use a device with proprietary drivers not supporting Linux

For me, it works flawlessly, since I avoid both cases.

as AI requires larger and larger data sets and processing power (energy, money), how will public domain catch up to their wealth accumulation? it's not like linux where chipping away at functionality and incremental UX adds up sufficiently over time

My guess is that such "digital narration" is on the brink of becoming available as a service to authors and publishers, with Amazon and Apple trying to get ahead of it by selling a product that can only be published on their platforms then. They are surely able to undercut human narration, but even on digital narration they have an unfair advantage of earning a share on every sale as well.

Will be interesting to see how this develops. Either independent digital narration becomes competitive enough that a publisher simply gets it done once and then sells it on all platforms, or this new platform-exclusive model is so disruptive that it becomes even less economic to produce a Audiobook, effectively making Audiobooks exclusive to Apple and Amazon/Audible (and whoever else has such a digital narration engine).

I'm curious what the economics are here, because getting someone to just read a book can't possibly be that expensive, right? I'm not talking celebrity voice-over work, but just getting anybody with a passable voice to sit down a spend a day or so reading a book into a microphone? Does that really cost that much more than having someone sit down a listen to the whole AI-generated audio book to do QA? And then there's all of the engineers who have to work on the project, the hardware to run all of it, etc. Seems like if it takes months to do and has to be QA'd anyways, it can't be that much more cost effective. Now, if it's completely computer-generated and can be a push-button feature? Sure, that makes sense. I wonder how close they are to that.

If you're talking about using a trained voice actor, yes, they will cost an hourly rate to do this, which is reflective of the care and training they put into their craft. One should also expect that they don't simply press record, and then do an entire read through. They will go back and try different takes on segments

If we're not talking about a trained voice actor, you may want to look at some thing like LibriVox, which is entirely volunteer run on public domain works, and while the efforts are appreciated, the quality is noticeably different.

You need to take into account that nobody can perfectly read a book first time. I've done a reasonable amount of short-fiction narration for audio magazines, and even with many hours worth of script reading under my belt I still fumble every third or forth sentence. So I need to re-read those, and then I (or somebody else) needs to find those errors and edit them out, replacing with the fixed audio. Then people need to listen to the entire file at least once to ensure the whole thing makes sense and I didn't leave out a sentence somewhere or something.

Audiobook narration is one of those things that is remarkably labour intensive, certainly much more than I'd have guessed before getting into it.

This was demonstrated vividly when Andy Serkis read The Hobbit unabridged live for charity then did a separate recording as a commercial release. Both were great, but you could see how many minor flaws were in the live recording that would have had to be re-recorded in the commercial version. (I’ve no idea how long it took to record the Lord of the Rings which he did a year later)

As an audible customer I can tell you clearly that I often buy audiobooks by a particular narrator (discovering new authors) because the narrator is so good.

I mean, Siri is quite good at reading texts (I imagine that's a huge training corpus) but I think we'll be in "uncanny valley" for quite a few years.

It's possible that the public just gets used to that.

Same! I remember being really impressed by Justine Eyre in something, and then being a little disappointed to find that 99% of her work is romance novels.

This is achievable with a very modest (say 50-100k) budget for voice actors and compute. Less if you’re happier with lower quality. Speech synthesis is probably one of the few areas in ML that’s trivially accessible to smaller orgs.

Even Stable Diffusion was only 600k which is hardly outside the reach of a startup. The only ridiculously expensive models reserved for the big end of town are the GPT3 etc language models, and I fully expect the data/compute requirements to come down considerably in the near future.

This is already easily achievable on consumer hardware, you can train something like Tacotron 2 + WaveRNN on your own computer to achieve similar, if not better results. Check out this repo:

https://github.com/coqui-ai/TTS

You can also clone someone's voice by finetuning a pretrained LJSpeech model and training a vocoder from scratch, I've had great success with as little as 15 minutes of speech.

Check my other comment in this thread, you might try our dataset. :)

Will check it out, thanks

> You can also clone someone's voice by finetuning a pretrained LJSpeech model and training a vocoder from scratch, I've had great success with as little as 15 minutes of speech.

Are you able to point to any articles to help get started with this please?

Unfortunately, I'm not aware of any beginner friendly tutorials.

The way I learned it was just by experimenting with various GitHub repositories (e.g. https://github.com/fatchord/WaveRNN or the one I linked earlier) but it takes a lot of trial and error. Might do a writeup at some point if I have time.

It really isn't though. The level of fidelity that Apple is demonstrating in those samples is very impressive. You can generate fine voices with little work using repos like that one, but to get to the level Apple has takes a lot of work.

EDIT: "fine voices" not "find voices"

Not from the companies selling audiobooks IMHO.

They will use this technology to save money on human speakers. If they release it into the public domain we'll end up with ebooks that can read themselves aloud and they'll lose part of the incomes from audio books.

My Samsung phone can read ebooks with one of Samsung's voices right now, but it does an awful job at pauses. Basically, no commas. With a good voice I could turn each one of my ebooks in an audiobook.

I don't really think I could listen to an AI voice reading a book. Perhaps some technical stuff, but not fiction. Even the difference between a mediocre vs good voice actor is huge, and can mean the difference between finishing an audio book or stopping after a chapter.

Edit: to be very specific, a really good voice actor will take on different voices depending on which character is speaking, and will act out scenes realistically. I honestly can't imagine any AI being able to do that.

There is https://commonvoice.mozilla.org/en though I’m not sure where and how is it being used.

Common Voice is more about building a dataset or how people talk, especially with accents.

While there is/was a voice synthesis project at Mozilla it was rudimentary like 3 years ago

You can play with it on https://uberduck.ai/ and they have a very active Discord!

What exactly is "Open Source" about uberduck? It looks like a proprietary tts saas to me; no links to a git repo and the "developer" section just shows how to get an API key and hit their service.

Reply on news.ycombinator.com