Hacker News Reader

Comment by kmnc | Hacker News Reader

Very impressive... but it seems like the AI relies entirely on abusing blink stalkers which with perfect micro is basically impossible to counter. It is no surprise it can crush pros when it has perfect timing and zero mistakes in using these units.

I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed (Micro harrasing early, early expansions, a very good understanding of when to attack/retreat). It looks just like your average pro..until you see its god tier micro.

Well, after seeing Mana crush it in the live game it seems the AI had zero clue as to what to do... it seems like it calculated it couldn't win a fight so let Mana destroy its entire base. So, just like with the Dota AI we see pros can exploit and win easily once they play around the micro advantages.

View on HN

It had a variety of strategies, not always making a lot of stalkers. And humans can do some really impressive blink micro too up to a certain number of units.

So that's one aspect of it but isn't the main strength.

Those were not one and the same agent. If you look at the figure they released on their website the given agent would probably always have gone for a lot of blink stalkers.

I was amused to see on their website that one of the top-rated agents we didn't get to see built almost nothing but Void Rays.

We're talking about the whole package of the AI, what the entire system is capable of, so that technical detail doesn't affect kmnc's point or mine.

The biggest effect is that it's harder to change strategy midgame, which is not all that critical.

It makes the trained agent much less like a human, individual agents are not creative, they‘ve just learned to execute a certain build order. It might turn out that this is enough coupled with flawless execution and micro.

"A human agent isn't creative, it just has a set of initial build orders it developed and picks one before the game starts, and after that it adapts to the situation weighted by the units it prefers."

Also the execution of the build order is nowhere near flawless.

As the commentators mentioned, it's no use building units to counter your enemy's army (Immortals over Stalkers) when the enemy can control their army so much more effectively.

I have to wonder if future competitive games will need to take into account the abilities of reinforcement learning algorithms when releasing balance patches.

But there IS use building units to counter your enemy's army. In the last live match when Mana won, his immortal archon zealot composition was what sealed the deal in the end.

And his far superior positioning.

I mean, the AI in that last match couldn't see the whole map at once.

Fog of War was enabled in all cases, but the last match was them enabling the "Scrolling window" that humans are forced to look at the game with.

The AI in all of the other games could see and control all of its own units on the map simultaneously. No human has this ability due to the limitation of the screen.

But they did say that the way the AI focused was roughly equivalent to moving the screen every couple seconds, not very different from pros, and that adding the limit didn't affect its performance against the old version.

Stuff like that is why StarCraft is not a very good game to test AI on. It's good for publicity, because it is well-known and has a pro scene so you can claim to "beat humans", but it's too complex in some ways (ruleset) and too simplistic in others (mostly just killing stuf). APM and micromanagement are huge, while long-term strategies are fairly limited compared to many other strategy games.

With all of this, it's really hard to see how much/which parts of the game the AI really "understands".

I'd love to see an AI that can play MAX or MoO from raw pixel inputs, even if it sucks at it.

There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter. And in the demonstration they filmed today, they actually limited the information the AI knows about to the screen space, which is probably along the lines of what you were wanting.

Their action counter rationale seemed fallacious, allowing 300 actions per minute sounds like it would give the bot an edge, presumably the bot has a much better ratio of meaningful actions vs all actions.

>There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter.

Which is exactly why StarCraft is not a very good game to test AI on. It's absurd to put arbitrary limitation on something to make the game "fair" and then pat yourself on the back simply because the algorithm won. If it can already win through pure micromanagement, why

There are tons of strategy games which don't revolve around micromanagement where APM simply doesn't matter. All turn-based games, for example. Or real-time games where building stuff is more important than combat.

> All turn-based games, for example.

You mean like chess? And go?

I think turning to a real time game with a complex rule set after showing they mastered turn based games with simple rule sets was very sensible.

> Or real-time games where building stuff is more important than combat.

Can you name one that is played professionally (important for balance and comparison to humans) where this is more true than starcraft 2? I think of starcraft 2 as very macro focused as games go.

I used to be ~80th percentile in North America (worst region) and I'm certain at the time any pro could have beat me without clicking anything outside of their own base (except the minimap).

>You mean like chess? And go?

What a clever reply. No, like the ones I named in the post above.

>Can you name one that is played professionally (important for balance and comparison to humans)

If you pause to think about it, things that make for a "good" pro scene are exactly the things that make computer games less than impressive for innovative AI research.

It looks like a human/AI centaur[1] would far exceed the abilities of either alone. In chess that lasted for about a decade, before the human became superfluous. In Go it seems to have happened already? But given the complexity of Starcraft I expect it to take longer than Go.

[1]https://en.wikipedia.org/wiki/Advanced_Chess

>I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed

I mean, the agents have learned the game from pro replays so to me it seems obvious it would evolve to play the same strats human pros use.

My understanding is that Alphastar is trained on reinforcement learning. I suspect this is unsupervised so it would have learned this behaviour independently without pro replays.

Per DeepMind's blog post[1], an agent was initially trained via supervised learning on pro matches. Then, the agent was forked repeatedly, as the population of agents learned via tournament-style self-play. So, while initial strategies could have been seeded by pro play styles, the final models were the result of models learning from games with other models.

[1] https://deepmind.com/blog/alphastar-mastering-real-time-stra...

Meaning that the final models evolved by playing against each other and they all started by using pro strategies, so, again, to me it seems kind of obvious that they would end up using pro strategies and in the best case just try to make them better

Ya, that one game was a bit... 'fresh'. But saying that it relies 'entirely' on abusing blink stalkers isn't quite right.

It did it in 1 game of the 5 they were showing.

Reply on news.ycombinator.com