DeepMind introduces ‘EATS’: adversarial, end-to-end approach to text-to-speech | Hacker News Reader

DeepMind introduces ‘EATS’: adversarial, end-to-end approach to text-to-speech

syncedreview.com

125 pointsYuqing745 comments

It seems like the abstract is a little bit misleading. WaveNet is almost scoring half a point better on a scale from 1 to 5. Tacotron 2 [1] is scoring even better and reporting a score which is almost human-like:

  EATS          4.083
  WaveNet       4.41
  Tacotron 2    4.53 
  Human Speech  4.55

It seems like a stretch to me to say it's "a comparable to the state-of-the-art models".

[1] https://google.github.io/tacotron/publications/tacotron2/ind...

They say it uses less supervision, so it's better in a different way.

You can't compare the scales directly across different papers because it's a subjective Mean Opinion Score (MOS) in 0.5 increments, usually taken from a different Mechanical Turk sample. It's a shame that the EATS paper didn't do their own MOS for many of the standard models. However, as pointed out in the paper the real benefits here are in reduced inference/training time and improved attention learning which many of the current models struggle with.

EATS and Wavenet are from the same research group. EATS and Tacotron have almost the same score for the human sample. None of the papers said that they used Mechanical Turk. Google has it's own internal MT-like service which is probably more consistent.

> more recently saw its self-taught agents thrash pros in the video game StarCraft II.

Clarification: DeepMind never fully 'beat' SC2 pros the way they did in Go.

Their AI was super impressive, certainly far beyond any other that has been developed. But they only went so far as consistently doing 'okay' against top tier players on the ladder, and that with some significant caveats to their performance. In particular, it's debatable to the extent that they're overpowering some players through sheer APM, rather than superior tactics or strategy. Yes, this is true even after they put in certain restrictions.

And given their recent silence on the issue, it looks like they may have given up, which would be unfortunate.

For those who didn't know (like me!) APM is Actions per minute... Seems as though the AI should be limited on this if it's to truly be outplaying humans; we all know computers are good at doing many many repetitive tasks extremely quickly.

It was limited, but the problem is that it was limited over various windows of time. The bot was able to execute short bursts of extremely high APM within those windows by being slower during non-peak moments.

That makes it more impressive but I guess if it did learn to be efficient like that it optimised to the rules. Although, to be fair, human players seem to be super human in this statistic - I was thinking about maybe 1-5 actions per second but the world record is 818 APM === 13.67 actions per second!

Also, its interface with the game is simply more efficient. The APM managed superhuman control without using control groups; for a human player, not using control groups at all would be an enormous handicap.

The implication here is that the AI can select units and buildings arbitrarily much faster than humans can, and doesn't need to spend APM on creating and maintaining control groups themselves.

No, it's not true. If you followed closely their APM was hamstringed to the point of being lower than players in many cases. In many of the playbacks I've seen it is the strategy that helped them win. Though when they lose against grandmasters it is often obviously poor decisions caused by a rabbit hole of thinking.

The one thing I was really wishing they'd demonstrate but that I don't think they did was a functional understanding of hidden information, ie. natural scouting and positioning to catch hidden moving units that you've seen glimpses of. I didn't see anything of that in the games, the AI won afaict largely on a superior understanding of economy.

In the game after they introduced camera movement speed limits the AI lost pretty badly to warp prism harrasement. It reacted like a noob (moving the whole army back and forth instead of splitting it).

That was a version that was trained for shorter time, though.

I'd say AI won mostly because of better micro and less mistakes, but I haven't seen the most recent games, maybe that changed?

This isn't true. Alphastar achieved super-human APM, in excess of 1500, in short bursts. Even if it's average APM was lower than a human this is still not a good comparison because it's value-per-click (e.g. not misclicking) is going to be extremely high. It's victories were largely due to superior micro control and a highly effective method of judging it's relative power in an engagement(i.e. when to push an engagement versus retreating). I think this says more about Starcraft II as a game than it does about the learning ability of DNNs. When you have perfect Stalker micro, nothing else in SC2 really matters (to some zero-th order approximation).

Human players tends to maintain high APM even in "idle" situation like the beginning of the match in which you don't have many things to do. They do it because it's easier for an human to have constant APM instead of spiking in critical situation like combats. I think that this pumps the average APM of a human player. Example of human player playing: https://www.youtube.com/watch?v=bexWuHmV32A

The APM limit was averaged over longer periods of time. So AI "saved" the APM during boring parts of the game and then used it up all in a short bursts of over 1000 APM during crucial battles. It was especially visible during the blink-stalker Protoss vs Protoss against MaNa (very micro-intense matchup).

Blink stalkers are very cost-effective units that are balanced by being very hard to micromanage perfectly. It's basically impossible over large area (during battles that take more than 1 screen) because it's pseudorandom which of your stalkers is targetted and needs to blink in any given moment, and you need to move the screen to that place to blink the stalker in time. AI knows health of all units without the need to move the screen to see it so it knows where to move the camera in advance.

AI perfectly micromanaged full limit army of blink stalkers during a battle that was 3-screens big. No human could physically do it with the current starcraft UI.

It's like chess pieces were 100 kg each and AI was steering an industrial robot :) Interesting, but it's mostly measuring stuff that's not about AI :)

In the next game MaNa went into immortals (units that specifically exist to counter stalkers in pvp), and still lost, because AI blink stalker micro was just so impossibly good.

>> The APM limit was averaged over longer periods of time. So AI "saved" the APM during boring parts of the game and then used it up all in a short bursts of over 1000 APM during crucial battles.

To be fair to AlphaStar, figuring out that it's a good idea to do this is not trivial. You could say it learned a kind of exploit (of the artificial restrictions placed on it).

AIs tend to find exploits of the rules instead of actually doing what we want them to, it's pretty common.

I suppose that is a part of the human understanding of neural networks. When you take out the human perspective of humans, we do pretty much the same thing.

It is true. I'm using APM as a shorthand, here, for mechanical aptitude.

Yes, the APM, in terms of raw numbers, was constrained. But the computer used its APM in a completely different way, one which players could not possibly replicate.

One obvious example: the AI doesn't use control groups. To anyone who's played Starcraft in a somewhat competitive fashion, that's a HUGE tell. Because the implication there, is that the AI doesn't need to make control groups, because it can just arbitrarily select whatever subset of units it wants to control, whenever it wants. Humans can't do that.

IIRC, the AI also tended to 'burst' its APM in the middle of fights. That's the opposite as what happens with human players. Human players often have higher APM outside of fights, when they're doing repetitive, simplistic actions (e.g. Zerg players slamming the "make lings" or "make drones" hotkeys on all hatches simultaneously) and/or spamming to keep up a rhythm. When they go into a fight, where they don't want to waste time on spamming and where commands to improve their position are less obvious, they actually tend to slow down.

Here's what one of the pros who played against AlphaStar had to say about it:

"MaNa: I would say that clearly the best aspect of its game is the unit control. In all of the games when we had a similar unit count, AlphaStar came victorious. The worst aspect from the few games that we were able to play was its stubbornness to tech up. It was so convinced to win with basic units that it barely made anything else and eventually in the exhibition match that did not work out. There weren’t many crucial decision making moments so I would say its mechanics were the reason for victory."

The AI was really good at determining when a fight was winnable/unwinnable, I think. That was the most impressive part for me, usually AI's for BW/SC2 were awful at knowing when to push vs retreat. AlphaStar turned around on a dime when the human player had a better army.

Speech samples in the DeepMind publication post: https://deepmind.com/research/publications/End-to-End-Advers...

I think this is a better link and should have been posted instead. These sound samples are pretty wild! Scary accurate.

"Ablation: No Monotonic Interpolation" is my favorite — wait for it...

I ran into that with tacotron sometimes too

Do I still hear a bit of 'autotune' in there, or am I projecting?

I wonder if DeepMind will tackle emoting; they certainly have something good enough for voice overs in a business context.

Thanks! You can also experiment with DeepMind's WaveNet using GCP TTS Service (free tier):

https://cloud.google.com/text-to-speech

What's the state of the art in TTS that might conceivably be runnable on a modern, offline mobile device?

WaveRNN. https://arxiv.org/abs/1802.08435

Is there any implementation? I'd love to be able to run something on my computer and generate speech.

The closest open source project is probably https://github.com/mozilla/LPCNet/

It seems that contrary to their sota Baseline, at least for English, text to speech is a solved problem (on their metric MOS) https://paperswithcode.com/sota/speech-synthesis-on-north-am... The SOTA has a 4.526 MOS vs their 4.084 and baseline of 4.41 and human performance is 4.55

They don't seems to have improved the state of the art :/ But it might be a promising new research direction.

https://paperswithcode.com/paper/end-to-end-adversarial-text...

It's crazy that Google is doing all that research and giving it for free to all their competitors.

So is Baidu.

The whole industry benefits from this sharing of knowledge.

Also, the best aren't going to work where they can't publish.

Also true: Google doesn't really have competitors. It only justifies their monopoly.

Nlp tasks are not solved yet and without this collaboration between big corporations progress would be far slower, even for Google. For example a chinese student has outperformed and obscoleted BERT (the Google best Nlp model) with XLnet. Conversely XLnet wouldn't have existed without the BERT architecture

I think their logic is 3 fold:

1) Everyone benefits greatly from collaboration across companies and researchers at univiersities 2) There is a lot of value for google (and everyone else) as a company to show their achievements for many reasons, such as attacting talent, partners and so on. 3) And lastly, they don't expect the secrecy of their research to accomplish much, i.e. someone might re-discover similar results not that far into the future

Arxiv pdf link : https://arxiv.org/pdf/2006.03575.pdf

I love the block diagram of the thing.

amazing as always, the current home tts is pretty good now as well

What is 'current home tts', is it any kind of TTS model you can run on your computer?

I'm wondering why MacOS which used to have a superior set of TTS voices didn't implement any of the new neural voice engines. It's been years since Tacotron and WaveNet, and still the same crappy system voices.

Speculating, but perhaps they meant "the current Google Home TTS".

yes, thanks for the clarification :)

Could that be applied to speech to text too?

Reply on news.ycombinator.com