Hacker News Reader

Comment by causality0 | Hacker News Reader

It's interesting that TTS is getting better and better while consumer access to it is more and more restricted. A decade ago there were a half dozen totally separate TTS engines I could install on my phone and my Kindle came with its own that worked on any book.

View on HN

This reminds me of voice dictation history. I remember using dragon and other upstarts, on a 486dx notebook. I could discuss with the laptop, ask instructions, receive answers, and control verbally all options, actions, and programs. At the same time, do dictation in different fields of study - training required in geology words or engineering words for example - which was remarkably accurate. So you could turn on your notebook; start running programs, verbally type (dictate) a report , save, format, print, it email it - then turn on a music player or watch a movie. All without touching a mouse or keyboard. This was in a busy room, with a party by the way. Everyone said, just wait until computer speeds are faster! But all the software was bought out by Apple, Microsoft. It didn't get improved on for over 20-25 years, and still not really f functioning. (Except siri Google etc.) the

In case it's of interest, when I last explored this topic in terms of the Free/Open Source ecosystem I was very impressed with how well VOSK-API performed: https://github.com/alphacep/vosk-api

Here's another project that builds on top of VOSK to provide a tighter integration with Linux: https://github.com/ideasman42/nerd-dictation

I agree so much, that I've started learning ML to make a decent opensource many-languages TTS working on smartphones.

But really, the situation is pretty good, with a lot of code and dataset available as opensource. Notably, if you're not constrained to smartphones and the like, you can run on your computer quite a number of modern models, see for instance https://github.com/coqui-ai/TTS/ (which itself contains many different models).

The work that needs to be done is """just""" to turn those models into something suitable for smartphones (which will most likely include re-training), and to plug them back into Android's TTS API.

If you've not already encountered them I'd definitely encourage you to check out these Free/Open Source projects too:

* Larynx: https://github.com/rhasspy/larynx/

* OpenTTS: https://github.com/synesthesiam/opentts

* Likely Mimic3 in the near future: https://mycroft.ai/blog/mimic-3-preview/

Larynx in particular has a focus on "faster than real-time" while OpenTTS is an attempt to package & provide common REST API to all Free/Open Source Text To Speech systems so the FLOSS ecosystem can build on previous work supported by short-lived business interests, rather than start from scratch every time.

AIUI the developer of the first two projects now works for Mycroft AI & is involved in the development of Mimic3 which seems very promising given how much of an impact on quality his solo work has had in just the past couple of years or so.

To the Kindle thing, I wonder if the rise of audiobooks as a product killed the latter off to remove competition.

It was a rights issue. The Authors' Guild argued that TTS required audio rights. Here's an article from 2009: https://www.theguardian.com/technology/blog/2009/mar/01/auth...

Just one more example of copyright stifling progress, innovation, and accessibility. The Authors Guild doesn't screw over the public by exploiting our insane copyright laws as often as the MPA or RIAA, but they've had their moments.

https://www.eff.org/deeplinks/2005/09/authors-guild-sues-goo...

https://www.cnn.com/2011/09/13/tech/web/authors-guild-book-d...

https://popula.com/2022/01/22/what-kind-of-writer-accuses-li...

Or publisher could license the audio right from the content creators, so they get compensated for the increase in profits that the tech/publisher is making?

If it's audiobook, then yes, it's sort of a performance and you could argue they need different licence. But with tts you basically create the audio with your own means on your device. But you still bought the book, so there is you profit. If I read a book to my child, would you want me to buy audio license?

I know it’s a rhetorical example, but given how stingy the copyright regime is: yes, yes they would. You maybe could manage a 20% off credit for the child’s listening rights.

Whats the difference between Audiobook and realtime text to audio translation, isn't the outcome/experience the same?

The difference between you reading and the software is one is 'mechanical', whether you want to constrain someone doing that in the rights you grant is debatable.

Copyright enables you (gives the creator the 'freedom' to choose) to make such choices, its what people choose to limit is the problem, as they tend to be very stingy.

> Whats the difference between Audiobook and realtime text to audio translation, isn't the outcome/experience the same?

The "outcome" (text is read aloud) is the same if you read it aloud to yourself. Really the difference is that when a publisher releases an audiobook they hire someone (sometimes multiple someones) often an actor or the original author to sit in a recording studio and recite. They pay for things like studio time, sound engineering, editing, the narrators time, sometimes music or foley, etc. It's very much a different product.

If you have a book and you recite it, or if you pay someone to come into your home and read it to you, or you get a bunch of software and have your computer read it for you, that's your right and at your own expense in terms of time, money, and effort. Some kind of text to speech software is expected on pretty much every device. Including such features in devices or using those features (especially the accessibility features) of your own devices isn't copyright infringement, shouldn't open you up to demands for payments from publishers, and is in no way comparable to a professionally produced audiobook. Maybe one day the tech will advance to where a program can gather the context needed to speak with and convey the correct emotion for each line and will be capable of delivering a solid performance, but right now we're lucky if more than 2/3 of the words are even pronounced correctly and the inflection isn't bizarre enough to make you question what was being said or distract you from the material.

As a thought experiment, how about if some new fancy feature/AI could take your book/text and generate a movie from it. The technology/publishing platform only licensed the right to publish the text/book, but they now have a way that you can watch a movie (in the past they had to get actors, director, sometimes multiple, pay people, studios etc)...

As a follow up it would be cool, TTS did use different voices for Narrator and characters in a book ... if someone patents that your welcome!

I'd guess that even a movie, made by AI using nothing but a book you paid for, would still be okay for personal use. Again, we already have the right to hire an entire theater troop to come to our homes and perform whatever we want. As long as you weren't making your AI movie commercially available you'd probably be alright, that kind of thing might be transformative enough to be covered under fair use, although I'm guessing somebody would still object to you posting it online for free.

Honestly, if AI ever gets good enough at crafting films from literary source material that the AI movies has any chance of competing with a hollywood production the entertainment industry is screwed. I'm positive that by then whatever crazy stuff that AI is putting out will be everywhere and playing with it would be way more fun than a movie theater ticket.

Different voices would be cool, but who is speaking which line can sometimes be ambiguous even for human readers. I'd be happy with just one voice that didn't sound like a robot or like a human voice spliced together from multiple sources.

> convey the correct emotion for each line and will be capable of delivering a solid performance

Yeah ... so you might be getting into performance rights then ;P

One of the problems is if TTS becomes comparable to an actor's performance. It's interesting that we are also now moving in the direction where you can copyright your voice.

yeah, actors are going to have to secure all kinds of "likeness rights" or something to prevent all kinds of media being made with them, but without them. There have already been cases like this: https://www.theverge.com/2013/6/24/4458368/ellen-page-says-t...

I still expect it'll mean a lot of very amusing outbound voice mail messages.

Oh noooo. There is a huge difference in quality. Voice actors are severely underrated. They can make or break an audiobook.

We recently listened to How To Train Your Dragon instead of reading it to our kids ourselves specifically because it was narrated by David Tennant.

Next you will need audio rights if you want to read a bedtime story to your kids...

Oh absolutely, the same way Amazon removed physical buttons so they could use them as leverage for their overpriced Oasis reader. My twelve year old Kindle 3 has TTS, page turn buttons, and a headphone jack, and it cost a hundred bucks.

Fun fact: If you try to use Siri to read the text of a website to you ("speak screen"), it refuses to do so if you are connected to a car's bluetooth.

One of the vast litany of non-AI related Siri flaws that make you wonder how a $2t company can achieve such tragic levels of incompetence.

That sounds intentional, like trying to prevent "public performances".

I wonder if Google has the same limitation. I use that functionality quite often but I do it over the aux input.

Your average mobile processor doesn't have anywhere near enough processing power to run a state of the art text to speech network in real-time. Most text to speech on mobile hardware are stream from the cloud.

I had a lot of success using FastSpeech2 + MB MelGAN via TensorFlowTTS: https://github.com/TensorSpeech/TensorFlowTTS. There are demos for iOS and Android which will allow you to run pretty convincing, modern TTS models with only a few hundred milliseconds of processing latency.

Dr. Sbaitso ran on a modest 386. Mobile device processors generally eclipse that and could definitely generate better quality TTS.

Not only is state of the art TTS much more demanding (and much much higher quality) than Dr. Sbaitso[0], but so are the not-quite-so-good TTS engines in both Android and iOS.

That said, having only skimmed the paper I didn’t notice a discussion of the compute requirements for usage (just training), but it did say it was a 28.7 million parameter model, so I recon this could be used in real-time on a phone.

[0] judging by the videos of Dr. Sbaitso on YouTube, it was only one step up from the intro to Impossible Mission on the Commodore 64.

Ok, I get it, state of the art TTS uses AI techniques and so eats processing power, buuuuut seeing that much older efforts which ran on devices like old PCs, the Amiga, the original Macintosh, the Kindle etc. used much less CPU for speech that you could (mostly) understand without problems, it may be worth exploring if it's possible to write a better "dumb" (i.e. non-AI) speech synthesizer?

Better than the ones those systems already have? I assume they’ve already got some AI, because without AI, “minute” and “minute” get pronounced the same way because there’s no contextual clue to which instance is the unit of time and which is a fancy way of describing something as very small.

I'm still hoping that a human being can tell which of the four possible ways to pronounce the name of the English post-punk band, "The The".

https://en.wikipedia.org/wiki/The_The

https://www.youtube.com/watch?v=orIy18qIaCU

I have a soft spot for the Yorkshire pronunciation: https://www.youtube.com/watch?v=lzymb0YJp7E&t=160s

The parent didn't mention real-time as a requirement. Offline rendering would well suffice.

28.7 million parameter is nothing for inference

Often you can prune parameters as well. You might be able to cut that down by a factor of 10 without any noticeable loss in accuracy.

Well, that's no surprise, because governments/corporations (in fact, The Govporation these days) strive to keep the status quo of being way beyond the capabilities of the people, hence the monopoly on weapon-grade tools - and I seriously think that AI-produced speech that is indistinguishable from the human one is a weapon-grade tool.

Your answer is in deep conspiracy territory. The reality is in my opinion more banal: money.

Microsoft and particularly Apple used to make cutting edge (at the time) TTS available as a marketing gimmick, but then did not develop this further because TTS was only relevant for screen readers and people with impaired vision weren't really their primary customers. Then TTS made advances and companies started selling high quality voices at high price points. Now companies want to make as much money with it as they can. Improving "free" TTS for ordinary customers is not really a top priority of Microsoft and Apple. Moreover, since network speeds have increased, any improved end-consumer TTS will send the text to a server and the audio back, so that a company can collect and analyze all the texts and make money with this spy data. That's how Google's free TTS server works.

> Now companies want to make as much money with it as they can.

That's exactly what I meant when I wrote "...strive to keep the status quo of being way beyond the capabilities of the people".

Monopolizing the means of maximizing the profits to maximize the power - now that's some deep conspiracy territory, indeed.

> Your answer is in deep conspiracy territory. The reality is in my opinion more banal: money.

Honestly, are those really different? How is it not in their interest for money?

Can't expect any honesty here, my friend. Vested interest and latent stockholm syndrom prevails.

Reply on news.ycombinator.com