Hacker News Reader

Comment by nightcracker | Hacker News Reader

> After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human. In response, DeepMind began using a number of tricks to keep the trial blind and stop players spotting AlphaStar, such as switching accounts regularly.

So instead of fixing the unfairness... they tried to hide it?

View on HN

Funny how they put it as:

> When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human.

framing it like the agent was doing some sort insane play, when the reality is the main evidence of a player being alphastar (other than its garbage decision making) was the fact that it wasn't using hotkeys!

The correct terminology would be "control groups".

The configuration of control groups is visible in replays; AlphaStar's replay data does not have any control groups in it.

Though, via its API, it is able to select arbitrary groups of units from anywhere on the map.

Personally, I would have loved to see it work through control group management because I think that is something that is important and steals attention away from humans. But probably to the researchers, it is just annoying data to model that doesn't get to the "core" of StarCraft.

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border.

Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things that would give AlphaStar an unfair advantage.

> These extremely difficult/impossible things didn't really give an advantage.

One of the videos I watched compared APM (Actions Per Minute) with EPM (Effective actions Per Minute). AlphaStar always has them nearly identical, which would be (according to him) basically impossible for humans.

Of course it's impossible for humans. Humans are used to clicking multiple times to do an action. Some actions you want to take in the game are very important, failure to do them correctly can mean a lost game, for example moving units in combat properly. This means that it's better to spam click that action a few times to make sure that the button press is registered properly, because it is possible for button presses to fail due to hardware, software, and most often a coordination error. This means that essentially every player has a much higher APM than EPM, but it's really EPM that counts. APM itself is mostly irrelevant.

Hmm, seems like it might be interesting to add a slight error rate for AI clicking and see how it handles it?

You only need excess dexterity than your opponent, and only for a couple seconds, to decisively win a match in SC2.

Error rates, EPM and APM are all red herrings.

Right, one example I remember from watching the games back in January was stalker micro. The high EPM of the AI allowed it to micro them picture perfectly and blink every single one away right before it died. In doing so it won a battle against impossible odds, and the announcers even commented on how it was way beyond the capabilities of the best players.

That exact strat is the reason they implemented the APM limit, it was a cheese strat only a machine could implement.

What you said is a meaningful advtange.

Scrolling screen is a major action to gather info and control units.

If one can control near edge units without scrolling it gives more stable view and lower chance of making mistakes.

There's an in-game option to disable scrolling the screen with the mouse and doing it with only the keyboard.

> For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border.

That humans cannot reliably perform these actions because of the limitations of our corporeal form means that Alphastar has an advantage over a human player. Limiting APM isn't enough.

The question isn't whether the AI still has advantages, but whether it's an unfair one. The current restrictions were implemented with input from TLO, and seem to work quite nicely.

One could, of course, add more restrictions (like not noticing things on the minimap all the time...), but that's not what's it about anymore. At this point, micro is comparable to humans, and we can start to compare macro and strategy.

Fwiw, when I watched some of the replays, I was disappointed by AlphaStar. It's a very consistent player with few mistakes, but it isn't very reactive and definitely not inventive. Instead of switching tactics when things don't work out, it generally continues with the chosen strategy. Often, that's enough: a well executed strategy with few errors often wins, even if it wasn't optimal.

> a well executed strategy with few errors often wins, even if it wasn't optimal

I think this is hits the chord with why a lot of people are talking about this bot as cheating more than they are being outplayed.

It's like fighting an aimbot in an FPS. They arent outsmarting you, but the mechanical consistency is inhuman and you are unable to force mistakes. At best itll feel like playing someone on their best day.

The heart of gaming, and why starcrat2 in particular is popular, is finding ways to play around each others mistakes.

Honestly, they should reduce Micro to the point that humans can bully it a little bit. How does the AI handle finding new ways to trade units to its advantage against a mechanically stronger opponent? How does it try to bait the human into having to take fights with unfavorable units / strategy.. as opposed to beating through them with mechanic mastery

The issue with limiting it past what humans can do is that in the self play training they'd never encounter what humans could do. The benefits of trying to make the limitations somewhat human-like is that the self play optimizes around roughly what it will encounter when it goes against humans

May be make self-play asymmetrical so sides in the game would have different sets of limitations? So main agent would have subhuman micro and exploiter agents would have superhuman micro.

I don't think the comparison with aimbots is fair. Aimbots are more like a perfect micro. In the beginning (when they did their first show match), Alphastar was insane at that. Now the micro is not giving an advantage anymore.

Where alphastar shines, is in making a strategy work. It won't attack into something that looks dangerous. It won't forget to build more units. It won't be distracted on two fronts. (That said: Alphastar did some really stupid mistakes as well.)

I agree with you, though: it would be much more interesting, if Alphastar was actually handicapped in the micro (at least at a later point in the game). Then it would really need to find interesting strategies and counter opposing units with more cost-efficient ones.

>Now the micro is not giving an advantage anymore.

I guess I need to learn more about the details here. Robotic consistency is a significant advantage, so unless it is handicapped I dont think the result is much of a leap from what we learned by it being insane at micro.

You can feel consistency on your opponent and it is extremely intimidating. Usually, players are consistent at different things.. like my micro might be much better with marines than tanks. To have consistent human-level micro across all units is already a huge advantage.

I agree the aimbot comparison is excessive, but the hyperbole was good for getting the point across.

You could take an fps AI with aimbot, and give it 220 ms input lag to say it doesnt have advantage over players anymore - but its still going to climb to the top of the ladder system because it isnt going to miss. It is either advantaged at aiming or disadvantaged based on that 1 number

I think it's less about fairness and more about what's interesting? When the AI finds an advantage that makes its gameplay kind of boring, nerf it somehow to see what else it can do.

It's an AI system, of course it has some advantage, that's why we are building them.

The advantage here is due to artifice and not necessarily generalizable.

To play against alphastar, you have to opt-in. IIRC it states Alphastar will hide itself.

One of the things people noticed in replays was the lack of control groups and in the case of zergs, the ability to select larvas directly, which no player ever does. It could have been as simple as removing these quirks.

Nobody selects larvas directly? I guess I am a computer then. Way worse than Alphastar tho.

Ha! So I always did that in Brood Wars, but in SC2, I put each hatchery on a hotkey (0, 9, 8, etc.) and then just pick the units I want to queue up. No need to select individual larva any more.

I like to map tilda, tab, and caps lock to 0, 9, and 8 so that my left hand doesn't have to reach as far.

New players (like when I first played) are likely to select individual larva before they do tutorials or learn the hotkeys, but you're right that high level players would almost never select one directly. Maybe after a hatchery has died and there are still larva remaining?

No, it doesn't select the larvae by dragging, it just selects them instantly from the other side of the map. I'd guess it either has a hotkey for the hatcheries and press it and the larvae key instantly without it showing in the replay, or it has a cheat ability to instantly select larvae.

Its API allows it to select units arbitrarily from anywhere (regardless of camera). Camera vision only controls the specificity of data it sees for on-screen units.

Oh, interesting. Thanks for clarifying.

For the type of beast AlphaStar is, they do not sound like simple tasks at all.

I'm not sure this is unfairness. IIRC they put in a fair bit of effort to put it on a level playing field with humans by limiting APM and not allowing it to observe multiple areas simultaneously by spam moving the camera.

It might have some minor unfair advantage in terms of being able to click with pixel perfect accuracy, but they're marginal and from watching this project evolve, it's pretty clear that the strategic planning aspect of alphastar has indeed become better than humans.

"Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM."

This is worse micro than top human players.

Wasn't blink stalker micro pretty central to its playstyle as protoss, the race it was best at? Why would you say that is minor and marginal?

It can no longer do micro like that, I have watched most of its games several times and it never did any micro feat like that again. On the contrary its micro was often lacking compared to humans, it won mostly through very strong timing attacks and uncanny ability to pick when to fight.

> It can no longer do micro like that

Is that supposed to be an intention in design? I'd figure an AI could easily outplay a human if that weren't the case, given its inherent advantages (e.g. better accuracy, the ability to instantaneously prioritize which units to blink in/out).

For example, [this](https://youtu.be/pETcAm82vXU?t=322) game is one example where an AI could perform even better using such tactics.

EDIT: another commenter mentioned it would make the game too unfair, so it was an intention in design.

Just as an example of how extremely unfair it would end up being, flawless AI micro looks like this: https://www.youtube.com/watch?v=IKVFZ28ybQs

100 zerglings vs. 20 siege tanks. Without insane micro the zerglings barely kill 2 siege tanks. With insane micro the 100 zerglings mop up the whole army with ease.

It's fascinating & fun to watch, but if your goal is to make an AI that can out-think a human it's super not useful, either.

That was from the version of AlphaStar that they demo'd in the earlier show matches. From what I gathered in the articles over the last few days, they took in a lot of feedback from that and adjusted this to make strategies like that inhuman blink micro less possible...

"... the strategic planning aspect of alphastar has indeed become better than humans."

This is not true. Top-level human players take into account who their opponent is, what builds they've used in recent games, their proclivities, strengths, and weaknesses. In a 7-game match they'll intentionally use builds which (in order to deceive) appear the same but have very different effects.

AlphaStar has better strategy than other AIs, but that's a low bar. My young son has better strategic appreciation of the game, and easily points out the difference between good strategy and AlphaStar's next-level micro.

They may be marginal but at this level of competition the margins are where the differentiation lies.

Reply on news.ycombinator.com