Amazon Polly – Lifelike Text-To-Speech | Hacker News Reader

Amazon Polly – Lifelike Text-To-Speech

aws.amazon.com

137 pointsstevewilhelm100 comments

Open articleView on HN

Previous discussion: https://news.ycombinator.com/item?id=13072944.

For a service claiming to be "lifelike", Joey has totally phrased that question as a statement.

I agree, it doesn't sound "lifelike" at all to me. Compared to WaveNet[1] it's day and night.

[1] https://deepmind.com/blog/wavenet-generative-model-raw-audio...

The "babbling" samples here are really fun. Would make for a great "Sims" language.

I actually found that pretty unsettling to listen to, and verging on creepy. I felt like I was listening to a computer trying to pretend to be human.

Wow, that was actually impressive. A few of those were hard to distiguish from human speakers. I love how they included breathing and mouth-sounds as well, really brings it to life.

Could we drop “Lifelike” from the title? It's marketing puffery (and untrue, it sounded very robotic to me to the point I was convinced I'd heard the wrong audio file). The actual story is that it's Amazon's text-to-speech-as-a-service offering, not that it's particularly innovative in terms of sounding good.

The male english sample sounds like dragon speech from 10+ years ago, and the female sounded more robotic than others I've heard...

I don't see (or hear?) the advance here...

i thought you guys were being either hyperbolic or overly critical. then i heard the demo files.

98 upvotes? Huh?

How can you tell?

i don't get the response- my comment does not have 98 votes- thought it was some reference i didn't get, or something.

Probably the submission had 98 up-votes at the time the comment was made, not your particular comment.

Oh... oh ok... yeah that makes more sense. i get it now. Super derp moment.

I agree that the voice sounds pretty bad, but I think the 'lifelike' part comes from the intelligence part. I.E WA -> Washington, and 75F -> 75 Fahrenheit

Text-to-speech systems have been expanding abbreviations for decades, and it always backfires in some cases. For example, DECtalk would expand "Sun" to "Sunday" regardless of context, with the humorous result that blind people reading tech news in the 90s would often hear about Sunday Microsystems.

Edit: Yes, that example was unfair, because it's from the 90s (and actually, DECtalk was largely unchanged since the 80s, until it was ruined in the late 90s). Here's a somewhat more recent one: With the ETI-Eloquence engine, which was last updated in 2002, the string "2 Marketing" (which one might find, say, in a Wikipedia table of contents) is expanded to "March twond keting".

Hence the need for intelligence

Which it fails miserably at. They probably picked the one example that worked.

Someone linked another example here:

https://soundcloud.com/zack-bloom/amazons-new-text-to-speech...

And it pronounces 1970s (the decade) as one-thousand-nine-hundred-seven-ths.

You're not wrong, but that's still the best real-time TTS sample I've ever heard.

One blind guy I know opined in the late 90s that it's better to let the brain do the interpreting. But I understand that expansion of abbreviations makes TTS more attractive for mainstream applications.

I was looking around for samples, and this is the only sample I have heard so far. I think that sounds pretty good. The emotional tone is off, but it is a lot better than say, Siri (which I understand was designed to sound robotic to highlight the tech). But I'm also the guy who has had my MBP announce the hour and it has freaked out coworkers.

I don't know if this is crossing into uncanny valley territory. I think some of this is not being used to it.

Google Map's driving voice sounds subtly better than that voice sample. I'm looking forward to seeing if a similar service is coming for GCP.

I know a guy who's abbreviated middle name consistently gets pronounced as "street"

> the 'lifelike' part comes from the intelligence part. I.E WA -> Washington, and 75F -> 75 Fahrenheit

This would drive me nuts, as there's no easy way to predict what sort of transformations it would make. Basically, if it's not near perfect, it's arguably worse than no intelligence at all.

With our PBX stuff, I know it's dumb, so I'm already accustomed to giving it something I know it will read correctly, like "domain dot com".

> This would drive me nuts, as there's no easy way to predict what sort of transformations it would make.

I believe that's what the Lexicons are for within the service.

If all of the auto transformations are defined Lexicons, that can be edited/removed, that would be fine.

I read the post I was replying to as some sort of undefined, built-in, set of intelligent transformations.

For example, 75F-> 75 Fahrenheit would require that you make a Lexicon entry for every value...1F, 2F, 3F, etc. That's not really intelligent :)

Sooo cynical. This is way better than most TTS systems, including Google's. The only one I've ever heard that is better is DeepWave, and that suffers from lots of background noise.

Do you mean WaveNet? The noisiness is because it uses 8bit (mu-law) audio. 8bit is because WaveNet is computationally extremely expensive. It's pretty much proof of concept at this stage, not production ready TTS.

There is a faster variant on GitHub, called, of course, Faster-WaveNet.

Seriously? Compare Polly's English samples (particularly #2) in the original article with the Google's WaveNet samples:

Polly 1: https://d0.awsstatic.com/product-marketing/Polly/HelloEnglis...

Polly 2: https://d0.awsstatic.com/product-marketing/Polly/HelloEnglis...

WaveNet 1: https://storage.googleapis.com/deepmind-media/pixie/us-engli...

WaveNet 2: https://storage.googleapis.com/deepmind-media/pixie/us-engli...

There just isn't a comparison -- Polly sounds way more robotic.

Err you are agreeing with me. I meant WaveNet not DeepWave (sorry forgot the name).

I don't see how I'm agreeing with you when you said "This is way better than most TTS systems, including Google's."

WaveNet is Google's, and I'm saying that this is significantly worse.

So I take it that this isn't Amazon's version of Adobe's VoCo or DeepMind's WaveNet.

Joel sounds like Microsoft Sam cleaned his blocked nose. shudders

It's more like the opposite of being innovative (unimaginative?)... They bring a product that could previously achieve that locally to a remote server and the quality is not any better.... and they want to charge you by the character... no thanks Amazon... hopefully it's free with Prime.

One thing I always still notice about these "lifelike" speech models is that they still have random pitch variation that wouldn't be present from a real speaker—sort of a "warbling", similar to audio heavily compressed by some cellular realtime audio codecs.

I assume this is due to random variation in the pitches the speakers in the training data used to say given phonemes. Would this, then, imply that speech models for tonal languages like Mandarin don't share this problem, since people are speaking with prescribed pitches more often?

I was wondering why it sometimes sounded like bad compression artifacts! Thanks.

Sheer speculation, but I wonder if that could be related to the "checkerboarding" artifacts you almost always see in AI-generated images?

Here's a good writeup: http://distill.pub/2016/deconv-checkerboard/

Re: the checker-boarding artifacts, that sounds interestingly like deconvolutional NNs treat their samples, at each deconv layer, as independent, rather than taking the Rule of Addition of probability-weights (P(A ∪ B) = P(A) + P(B) - P(A ∩ B)) into account. (Would a model that did do this then be a "Bayesian Deconvolutional Neural Network"? Does that even make sense?)

I doubt this would result in audible artifacts at the audio level, though. The visual checkerboarding appears at the 1px detail-level of the output image—this would translate more to minor fluctuations along an audio sample's Nyquist frequency, which is effectively undetectable if the output sampling rate is of the usual kind (44.1kHz).

I always wonder why they don't program a second pass over the output with a model, say, trained on radio conversations. That second model just for audio 'quality'.

I'm not sure about Mandarin, but this is definitely true for Japanese. Near-human Japanese text-to-speech has been around a very long time.

So good they can even sing, and so well that they create entirely virtual pop stars: https://en.wikipedia.org/wiki/Vocaloid

Vocaloid still sounds robotic (at least, to me). It just so happens when you make autotune/melodyne'd pop songs, it really doesn't matter.

Vocaloid isn't automated though, it's basically an instrument that humans "play". The software has specific controls for pitch, aspiration, etc

True, but Japanese isn't really a tonal language, so it doesn't really evidence the parent's point.

Japanese isn't technically "tonal" but does have "prescribed pitches" which certainly affects the ease of text-to-speech.

Additionally when we speak we don't just 'replay' words; we add emotions and so on. Reminds me of this video [1] analysing how the actor Anthony Hopkins converts his lines to speech for his scenes.

[1] https://www.youtube.com/watch?v=4kSGkGKwp9U

The spoken word has so many nuances. Cadence, inflexion, tone, and then the facial performances are never captured by pure text to speech. You'd almost have to be able to set boundaries for seriousness, playfulness, innuendo, indifference, sarcasm etc because none of that is conveyed in the literal text without additional descriptions or contextual analysis of the conversation.

It'd be interesting to take the Joint Many-Tasks approach here, training some of an RNN's layers on text-to-speech, and then another set of layers on sentiment analysis of speech-audio, where the error of the sentiment layer can backpropagate into the TTS layer.

I guess you could do something like vocaloid does to better "shape" the sounds of each word. I guess its pretty tough to do that automatically though, since words by themselves don't convey the emotions that they should be portrayed with.

I mean, a word or phrase could very much change meaning depending on how its spoken (excitement, disgust, fear, sarcasm etc) and even if the context is available in the text (as is often the case), its rather difficult to machine-detect, at least, for now.

Southerns have an accent. New Jerseyans have an accent. British have an accent. Can't computers have an accent, too?

Wouldn't we describe what TTS systems do as more of a speech impediment? Sure they use a particular accent, usually that of a standard prestige dialect. But it's not the accent that's weird.

Chinese (Mandarin) text to speech still shares the problem of not sounding natural (sounds robotic), but it is a little better than English.

Example (click on the blue speaker icon): http://dict.baidu.com/s?wd=%E7%88%B1%E5%B1%8B%E5%8F%8A%E4%B9...

If you grab the training set for CMU-Sphinx, you'll see that the voice actors providing base input are trained to speak that way.

At a secondary level, I imagine there's a sort of pitch trajectory for a sentence, and if a word or syllable ends too far off that trajectory, the return to the pitch trajectory is fast enough to be easily detectable.

I don't know about tonal languages. But you're definitely right about the artifacts of concatenative speech synthesis.

It's worth noting that purely synthetic speech, while lacking the superficial lifelike quality, doesn't have these little inconsistencies. Perhaps that's one reason why many blind people, particularly power users, prefer pure synthetic text-to-speech engines, such as ETI-Eloquence (commercial) and eSpeak (open source). These systems especially sound better at high speeds than the newer "lifelike" ones.

Just for fun, here's a side-by-side comparison between a concatenative system, IVONA (acquired by Amazon):

http://mwcampbell.us/tmp/derefr_comment_ivona.mp3

And a purely synthetic system, Eloquence:

http://mwcampbell.us/tmp/derefr_comment_eloquence.mp3

The latter is the text-to-speech engine I use every day, and that clip was synthesized at the speaking rate I normally use.

Edit: It appears the word I was looking for when I said "purely synthetic" is "parametric".

How do you feel about the WaveNet results, by comparison?

• https://storage.googleapis.com/deepmind-media/pixie/us-engli...

Those phrases sound very natural indeed. But are they carefully selected demo phrases, for which the system might have been tuned? That's always a danger with marketing demos. So I have two questions:

1. How does that engine sound when reading arbitrary text at high speed?

2. Can I run it on my own computer, so I can have low latency in my screen reader? (That, of course, is a problem with Amazon Polly as well.)

They say they doesn't charge for royalties for using generated audio. This could be huge for accessibility and alternative forms of media consumption for articles and online media. Generate once, distribute to all.

The speech that's generated still sounds relatively artificial.

This is in comparison with Adobe's tech where it samples a real human voice and allows speech to be inserted (http://arstechnica.com/information-technology/2016/11/adobe-...).

There's a lot of negative reaction to Polly. Besides WaveNet (which is not available as an API-- and really slow), what are better alternatives? Or is the reaction just to Amazon's marketing speak using "lifelike" and "deep learning"? I find most of the voices to be okay, but Joanna is pretty good and better then anything I've heard besides WaveNet.

Also, with Amazon's FPGA investment how long before it implements WaveNet using FPGA's to reduce the speech generation time to something that is realtime?

Why would Google use FPGAs over its own (faster) TPUs? Chances are Google either doesn't have enough TPUs (it may be waiting on the next generation to go massive scale with them), or Wavenet is still too computationally intensive even for TPUs.

I am sure Google is looking at WaveNet & TPUs, I was just thinking Amazon would look at using its new massive FPGAs to implement WaveNet. I have no idea if the FPGAs would be fast enough.

English speakers sound brain dead to be honest. This sounds no different from pre-NN attempts, only difference this time is it's behind an Amazon paywall.

Here's a sound clip of one of the voices I pulled, if you'd like to hear it: https://soundcloud.com/zack-bloom/amazons-new-text-to-speech...

The demo files in the article sound pretty decent, but this sample is pretty bad. It sounds pretty robotic by itself (lots of artifacts, odd pitches, no pauses), but saying "during the one thousand nine hundred and seventies" is the real kicker.

I'm a native Portuguese speaker and I can say that this is far from how people speak Portuguese. It sounds like pretty much other TTS solutions.

However, Spanish and English are awesome and really sound like a person. This is great!

Edit: After playing a little bit with it, it does sound better than anything I have seen before even in Portuguese. I believe that the weirdness is just for the standard/demo phrase. When I tried some custom phrases (curse words, mainly) it does sound very natural.

Also the sample text in Portuguese has a bad construction

Given that this supports Speech Synthesis Markup Language [0], I wonder what the impact will be on positions that talk for a living. While this is unlikely to replace voice actors for games/cartoons, I wonder if public service announcers and such will be replaced by this robot.

https://www.w3.org/TR/speech-synthesis/

I really like text2speech because I really like the computer to read articles to me while I'm doing something else, but I won't use any internet service for that just to get a little bit more quality in comperison to Apples "Alex" or MeryTTS http://mary.dfki.de/

Me too. I usually play Reddit and HN on Alex voice. It's the best instantly accessible voice on Mac, even though it is quite old.

#offtopic a bit but i wonder if anyone knows a good api for the other-way around -> speech to text

Shoot - I'd love an API you pass in txt in one language and get multiple languages back in txt and speech.

Amazon Lex and Google's GCE offer this.

Hello

I've never tried it myself as an api but a quick poke around the googles throws this out https://cloud.google.com/speech/

Shameless plug: check out Tropo[1]: you can interface with SIP.

One thing to keep in mind regarding speech to text: if you need accuracy, it helps immensely if you can guide the recognizer with a list of expected answers. Un-hinted transcription will do its best to recognize words but without AI to understand the context, there are still quite a few possibilities for mis-recognition. More info about these two here[2].

[1] https://www.tropo.com

[2] https://www.tropo.com/docs/voice/transcription-vs-speech-rec...

(Full disclaimer: I'm in the Tropo BU)

Amazon Lex? https://aws.amazon.com/lex/?nc2=h_l3_db

The thing these voices always miss is context. We need about 700 variations of every textual sentence that all depend on the mood or goal of the conversation and modulate pitch, volume, speed, word choice, etc. appropriately.

This experiment does cover the "generic computer voice" role quite well, but customers don't want that on e.g. their website. I would want a "I'm a professional designer" voice. So there will always be markets for more appropriate voices [*]. But this one does sound very nice at what it does.

The same will happen with robots, btw. Once we get a generic humanoid robot that can do everything, we will still seek aesthetic variations and employ remote body actors.

If you want to play around with it..

https://console.aws.amazon.com/polly/home/SynthesizeSpeech

Those samples are absolutely terrible.

The idea of applying machine learning to add natural emphasis to speech is a good one, but seeing how language machine learning systems like Google Translate fail on very basic contextual cues, I don't have much hope... Especially when their demo samples are on par with systems from a decade ago.

I'll stick to Bruce.

Congrats to the team behind the release in Gdańsk, Poland!

That's great to hear! I'm from Gdańsk too :) Out of curiosity - any more info about the team behind Amazon Polly?

Probably the Ivona team, no? https://www.ivona.com/

Yeah IVONA. Sounds similar and after acquisition Amazon was looking for engineers to build cloud offering :-).

Hah, that confirms my guess after noticing the voice names for Ivona and Polly are exactly the same :)

How is directly linking a .mp3 the best way to preview their technology? You'd think there would be a much more elegant solution to showcase Polly on their site.

A little bit on the underwhelming side, specially given the "lifelike" moniker. But it seems pretty useful otherwise. What are the competing services?

In case anyone would like to hear what it sounds like: http://vocaroo.com/i/s1mM3ExvMuwC

To play around with it: https://console.aws.amazon.com/polly/home/SynthesizeSpeech

Interesting to hear Danish and Icelandic. Does the language affect how difficult it is to get realistic sounds?

I assume now you can use LEX + Polly to make your own Echo.

Google, IBM and Microsoft all have invested way more in AI already and are actually way ahead of Amazon technically. However Amazon now becomes the first one that makes these technologies really approachable for ordinary developers. Way to go Amazon!

Hasn't Google had a Text-to-Speech service (for e.g. the speaker button in the Google Translate page) that's been around for aeons?

I remember it sounding pretty life-like. Don't know if they expose an API for it, and sell it though. (It would be great if they did.)

For anyone interested in using something like this to listen to articles/text, you should check out https://narro.co

I'm curious what some of the industrial or business use cases are for a service like this. I'm not saying there aren't any, I just wonder who this scratches and itch for.

I'm more excited about the html5 tts API than these services when it runs in the browser the cost is $0. The costs if these 3rd party APIs add up and add latency to your app.

I'm sorry, but lifelike? Not yet. More like better robot.

The female Spanish speaker has a strong Scandinavian accent.

In our team nobody speaks English as first language. Now we have to hire actors for English voice narration in our demo videos. If Amazon Polly gets better, we might be able to switch to synthesized voice narration and don't deal with actors anymore.

PS.That would be one more job lost to AI.

Does anyone know why there is a 1000-character limit?

It says

> The size of the input text can be up to 1500 billed characters (3000 total characters). SSML tags are not counted as billed characters. [1]

Not sure where you got 1000 from. As to the why - no. It doesn't seem like too much. 2 minutes of speech maybe in there?

[1]http://docs.aws.amazon.com/polly/latest/dg/limits.html

Danish is good, the rest sound not so very lifelike.

The Danish doesn't sound very good to me, compared to Apple's voices

Old style TTS quality + Marketing blablaing. It's more costly since you pay the blablaing.

Reply on news.ycombinator.com