Hacker News Reader

Comment by TulliusCicero | Hacker News Reader

> more recently saw its self-taught agents thrash pros in the video game StarCraft II.

Clarification: DeepMind never fully 'beat' SC2 pros the way they did in Go.

Their AI was super impressive, certainly far beyond any other that has been developed. But they only went so far as consistently doing 'okay' against top tier players on the ladder, and that with some significant caveats to their performance. In particular, it's debatable to the extent that they're overpowering some players through sheer APM, rather than superior tactics or strategy. Yes, this is true even after they put in certain restrictions.

And given their recent silence on the issue, it looks like they may have given up, which would be unfortunate.

View on HN

No, it's not true. If you followed closely their APM was hamstringed to the point of being lower than players in many cases. In many of the playbacks I've seen it is the strategy that helped them win. Though when they lose against grandmasters it is often obviously poor decisions caused by a rabbit hole of thinking.

The APM limit was averaged over longer periods of time. So AI "saved" the APM during boring parts of the game and then used it up all in a short bursts of over 1000 APM during crucial battles. It was especially visible during the blink-stalker Protoss vs Protoss against MaNa (very micro-intense matchup).

Blink stalkers are very cost-effective units that are balanced by being very hard to micromanage perfectly. It's basically impossible over large area (during battles that take more than 1 screen) because it's pseudorandom which of your stalkers is targetted and needs to blink in any given moment, and you need to move the screen to that place to blink the stalker in time. AI knows health of all units without the need to move the screen to see it so it knows where to move the camera in advance.

AI perfectly micromanaged full limit army of blink stalkers during a battle that was 3-screens big. No human could physically do it with the current starcraft UI.

It's like chess pieces were 100 kg each and AI was steering an industrial robot :) Interesting, but it's mostly measuring stuff that's not about AI :)

In the next game MaNa went into immortals (units that specifically exist to counter stalkers in pvp), and still lost, because AI blink stalker micro was just so impossibly good.

>> The APM limit was averaged over longer periods of time. So AI "saved" the APM during boring parts of the game and then used it up all in a short bursts of over 1000 APM during crucial battles.

To be fair to AlphaStar, figuring out that it's a good idea to do this is not trivial. You could say it learned a kind of exploit (of the artificial restrictions placed on it).

AIs tend to find exploits of the rules instead of actually doing what we want them to, it's pretty common.

I suppose that is a part of the human understanding of neural networks. When you take out the human perspective of humans, we do pretty much the same thing.

This isn't true. Alphastar achieved super-human APM, in excess of 1500, in short bursts. Even if it's average APM was lower than a human this is still not a good comparison because it's value-per-click (e.g. not misclicking) is going to be extremely high. It's victories were largely due to superior micro control and a highly effective method of judging it's relative power in an engagement(i.e. when to push an engagement versus retreating). I think this says more about Starcraft II as a game than it does about the learning ability of DNNs. When you have perfect Stalker micro, nothing else in SC2 really matters (to some zero-th order approximation).

It is true. I'm using APM as a shorthand, here, for mechanical aptitude.

Yes, the APM, in terms of raw numbers, was constrained. But the computer used its APM in a completely different way, one which players could not possibly replicate.

One obvious example: the AI doesn't use control groups. To anyone who's played Starcraft in a somewhat competitive fashion, that's a HUGE tell. Because the implication there, is that the AI doesn't need to make control groups, because it can just arbitrarily select whatever subset of units it wants to control, whenever it wants. Humans can't do that.

IIRC, the AI also tended to 'burst' its APM in the middle of fights. That's the opposite as what happens with human players. Human players often have higher APM outside of fights, when they're doing repetitive, simplistic actions (e.g. Zerg players slamming the "make lings" or "make drones" hotkeys on all hatches simultaneously) and/or spamming to keep up a rhythm. When they go into a fight, where they don't want to waste time on spamming and where commands to improve their position are less obvious, they actually tend to slow down.

Human players tends to maintain high APM even in "idle" situation like the beginning of the match in which you don't have many things to do. They do it because it's easier for an human to have constant APM instead of spiking in critical situation like combats. I think that this pumps the average APM of a human player. Example of human player playing: https://www.youtube.com/watch?v=bexWuHmV32A

The one thing I was really wishing they'd demonstrate but that I don't think they did was a functional understanding of hidden information, ie. natural scouting and positioning to catch hidden moving units that you've seen glimpses of. I didn't see anything of that in the games, the AI won afaict largely on a superior understanding of economy.

In the game after they introduced camera movement speed limits the AI lost pretty badly to warp prism harrasement. It reacted like a noob (moving the whole army back and forth instead of splitting it).

That was a version that was trained for shorter time, though.

I'd say AI won mostly because of better micro and less mistakes, but I haven't seen the most recent games, maybe that changed?

Here's what one of the pros who played against AlphaStar had to say about it:

"MaNa: I would say that clearly the best aspect of its game is the unit control. In all of the games when we had a similar unit count, AlphaStar came victorious. The worst aspect from the few games that we were able to play was its stubbornness to tech up. It was so convinced to win with basic units that it barely made anything else and eventually in the exhibition match that did not work out. There weren’t many crucial decision making moments so I would say its mechanics were the reason for victory."

The AI was really good at determining when a fight was winnable/unwinnable, I think. That was the most impressive part for me, usually AI's for BW/SC2 were awful at knowing when to push vs retreat. AlphaStar turned around on a dime when the human player had a better army.

For those who didn't know (like me!) APM is Actions per minute... Seems as though the AI should be limited on this if it's to truly be outplaying humans; we all know computers are good at doing many many repetitive tasks extremely quickly.

It was limited, but the problem is that it was limited over various windows of time. The bot was able to execute short bursts of extremely high APM within those windows by being slower during non-peak moments.

Also, its interface with the game is simply more efficient. The APM managed superhuman control without using control groups; for a human player, not using control groups at all would be an enormous handicap.

The implication here is that the AI can select units and buildings arbitrarily much faster than humans can, and doesn't need to spend APM on creating and maintaining control groups themselves.

That makes it more impressive but I guess if it did learn to be efficient like that it optimised to the rules. Although, to be fair, human players seem to be super human in this statistic - I was thinking about maybe 1-5 actions per second but the world record is 818 APM === 13.67 actions per second!

Reply on news.ycombinator.com