Google’s DeepMind Achieves Speech-Generation Breakthrough | Hacker News Reader

Google’s DeepMind Achieves Speech-Generation Breakthrough

bloomberg.com

87 pointsjrcii45 comments

Also https://news.ycombinator.com/item?id=12455510

This Bloomberg article adds nothing, and is less informative & interesting (doesn't have the audio samples for starters) than the original DeepMind blog post, and shouldn't be on the front page.

Here's the link to the paper: https://drive.google.com/file/d/0B3cxcnOkPx9AeWpLVXhkTDJINDQ...

And the WaveNet site with audio samples: https://deepmind.com/blog/wavenet-generative-model-raw-audio...

The comparison against state of the art Parametric and Concatenative methods are pretty mind blowing.

Particularly listen to the music samples. That's a generated piano piece that sounds quite musical.

They even include breaths and other auditory signals that really make for a convincing speech sample.

> Particularly listen to the music samples. That's a generated piano piece that sounds quite musical.

That blew my mind. I've heard computer generated music before, but it's been just synthesized with usual methods while the computer is just the "composer".

I find the crackling and buzzing a bit awkward, though. It's probably an artifact of the algorithm and can probably be mitigated with some simple filtering.

Thanks for the links. It is definitely an improvement from the previous state of the art, although it seems surprisingly similar to artificial methods. And by that I mean that a number of RNN based network solutions for things like image classification or speech recognition produce output that is very unlike the non-RNN previous solution. This feels like they were training the system using artificially generated speech so that it could automatically generate artificially sounding speech.

The speech samples suffer from common flaws in prosody. When it says "The Blue Lagoon is a 1980...", it omits the pause after Lagoon. When giving a definition, there should be a comma-length pause after the term. The pacing would be correct for a non-defining sentence later in the same article like "The Blue Lagoon is a popular subject for parodies, including ..."

Fixing that requires higher than sentence-level learning, since it has to know that it's introducing a potentially unfamiliar term which can only be known from the sentence's context within the article.

That is impressive. It seems like they're getting to the point where they can use a speech synthesis engine to train a speech recognition engine.

This raises some interesting questions. I've always thought that recording conversations would be sufficient "proof" of what someone said. It seems that soon audio alone will not be sufficient — video will be necessary as well.

Until their pixelCNN/RNN or some other generative model gets to the point where it can make video at sufficient resolution too.

you mean this?

https://www.youtube.com/watch?v=ohmajJTcpNk

Wow. Alright, can't trust anything anymore. That makes sense that they solved the problem more directly - how do I move someone else face in a video.

Nope

Do you remember a film titled Rising Sun?

A book by Michael Crichton

But that's just playing catch-up. Eventually video will be machine-made. It is hard to imagine given that need and the broader trends of society that we will someday NOT be living in a perpetual survellience state. However then the only way to ensure objective fairness is to make this information free and open to all. The second you close a door on access, corruption will explode.

Or, alternatively, we do not go that route and as a result the burden of truth is diminished to the point where factual information will no longer be enough to make a definitive judgement.

This Pandora's Box is staying open. Let's all hope for the sake of civilization that we use its powers responsibly.

Sorry! look at this video and restate your statement about "video". https://www.youtube.com/watch?v=ohmajJTcpNk

Wow!! That is amazing!

They used this technique in Mr.Robot to emulate Obama talking about the Ecorp hack

WOW

i figured knowing obama's interest in good tv shows, he might have done the cameo! this is really neat stuff

jaw dropped. Thanks for sharing.

holy shit

It'll be interesting to see once "true" voice emulators become main stream how it's exploited for good and evil.

Reminds me of "virtual" kidnapping scams where an attacker knows the victims phone will be unreachable and the attacker calls a relative demanding they wire funds or the victim will be killed. Attacker plays back a voice sample they've captured from the victim that makes it sound like they're in trouble an need help; basically the attacker calls the victim and say something like "May I help you?" repeatively until the victim responds with something like "No, I don't need your help!" an panicked voice - which is then edited to say "Help! I need your help! Help!"

On the fluff note: If this system is called WaveNet. The next weather prediction system based on deep learning should be called SkyNet.

SpaceX's server farm is called exactly that. Supposedly Elon hates the name...

I can almost hear him: "SkyNet is no joke, guys..."

Here are some fun samples for curious

https://deepmind.com/blog/wavenet-generative-model-raw-audio...

always wondered if something like this could be done. Now just train it on an famous individuals wav data and you got a free celebrity endorsement.

I think that something rather different may take place: celebrity voices will be copyrighted (or granted similar protection under law) by the individual themselves or their estate, and licensed for use.

David Attenborough, Morgan Freeman and Billy West: call your offices.

Having Mr. Freeman tell me which directions to take will definitely make road trips more fun.

This has been a (paid) thing for a long time: https://www.tomtom.com/en_gb/sat-nav/maps-services/shop/navi...

Recording: https://youtu.be/2ljFfL-mL70

How would you copyright it? Neural Nets have weights that mean nothing outside of NN. The samples could be copyrighted. Maybe you train it on a cross between an impersonator and the famous person. Wouldn't that be a derivative work? (Famous training samples, impersonator training samples, copyrightable quote)

Derivative works are usually subject to the copyrights of the works they are derived from. If you used samples you were not licenced to use, you'd potentially be violating copyright, but I think this is one of those areas where the law hasn't yet caught up to technology. As far as I know, no cases have brought this up yet.

Yup! There was a Planet Money episode about this, called Frank Sinatra's Mug. It's all about how Frank Sinatra had the laws in California changed in order to protect the likeness - including the sound - of a celebrity. Apparently it's Section 3334 of California's Civil Code.

Quoting from the episode: "Essentially what it says is that you can't take somebody's name, or their likeness, or their signature or their voice, and use it for commercial gain without permission from that person. And if you do that, you're liable."

The full episode is here - it's one of Planet Money's best. http://www.npr.org/templates/transcript/transcript.php?story...

What if they are trained on soundalikes' voices? The tricky part would be that you probably couldn't label it as the celebrity's voice.

Are you implying that there could be a market for synthesized knock-off celebrity voices? :D

The more I think about it even if you used originals you could licensed John Wayne's voice from Disney(or who ever owns clips of it) or something you couldn't imply there was an endorsement it would be fraud.

I want to see WaveNet used to create a synthetic William Shatner, trained on old episodes of Star Trek, TJ Hooker and priceline.com commercials!!!

Patrick Stewart and Morgan Freeman's voices should definitely be replicated.

Also

- Morgan Freeman

- Alan Rickman

- Liam Neeson

- David Attenborough

- John O'Hurley

- Denzel Washington

What's the point of authoring an article like this and not including samples so we can listen to it?

The original blogpost has samples and everything, resulting in a much better article

https://deepmind.com/blog/wavenet-generative-model-raw-audio...

Really this is natural text-to-speech.

16,000 analyses per second. So, given Moore's law, around the iphone 9?

Assuming it's done on device; you could stream it today!

The impression I get from the article is that it takes a supercomputer to generate it right now. So lets say 5 years? till it becomes mainstream.

90 minutes of computation for 1 second of audio on DeepMind's GPUs: https://twitter.com/hardmaru/status/773968758519902208

That's a lot of cranks of Moore's law. Better hope for considerable algorithmic improvements. (Raw is probably overkill anyway.)

54000 seconds! Let's say it halves every year. To get it to one second you need about 12 years. But if it drops by 1/3 every year, it would take 7.

Not stating anything especially mind-blowing. Just restating the shocking speed of exponential growth/shrinkage.

Reply on news.ycombinator.com