Hacker News Reader

Comment by philipkglass | Hacker News Reader

In any real-world application (robotic manufacturing, self driving vehicles, precision guided munitions targeting...) extremely fast reaction times are an essential and expected advantage of machines. I see AI game playing as interesting mostly because it provides a way to pit software against people without building a bunch of expensive physical infrastructure. If anyone ever builds a "real" robotic army it will surely use all the APM and TTFA advantages it can possibly achieve. In that sense, allowing the same advantages in combat simulation games is closer to a proper man-machine contest than one where those advantages are taken away from the machine.

View on HN

It comes down to what you are interested in. Right now DeepMind seems to be focused on beating humans on strategy and tactics, which is reasonable, since those are the contested areas and cpu winning on APM and TTFA has been solved for long ago. To not confuse one for the other, putting limits on the later (at some point maybe even stifling itself beyond what a pro player could reasonable to) seems like an excellent way to go about it.

> cpu winning on APM and TTFA has been solved for long ago

Has it? I'm not a RTS player but I thought that simple bots couldn't compete with top StarCraft players just with speed. It required strategy plus speed to first become competitve (though less strategy than required with slow APM and TTFA).

You are right, even average players can easily beat the AI player in Starcraft 2 (when the AI is not cheating)

Which AI are you talking about? The game-provided AI is not designed to beat players. It's designed to be a fun game mode for players.

That sort of game AI isn't designed to be strong. It's designed to be fun. See e.g. this talk titled "playing to lose" https://www.youtube.com/watch?v=IJcuQQ1eWWI

Individual mutualisk micro and perfect worker harass without decelerating are very very strong in Starcraft:Broodwar

Brood War is much more dependent on micro than Starcraft 2. Sean "Day[9]" Plott has mentioned this a lot in his videos. You can only select 12 units at a time and cannot select more than one building in Brood War. And there is a bit of latency in Brood War that doesn't exist in SC2.

One such video (skip to about 4:30 for opinions on mechanics): https://www.youtube.com/watch?v=EP9F-AZezCU

Apologies, I should have been more precise: A cpu will win when you start isolating and creating a scenario to test for APM and TTFA (obviously), so if you are interested in advancing strategic and tactical AI capabilities – basically out-thinking the human – it makes sense to cap those solved parts to make sure you are winning because of progress you make on your objective.

It's disputable whether any such limitations currently applied to AlphaStar are really working as intended. There has been some controversy over it. I saw the matches and it did seem that some of the actions taken by AlphaStar would not be possible to do by a human. Human APM is limited not just by the mechanical speed of one's hands, but also by the mental models that exist in one's brain. Muscle memory can't do everything for you. But in AlphaStar's case, it can perform every type of action as if it were done with the speed of muscle memory.

The original commenter's remarks about TTFA seems to be equivalent to what I'm referring to.

Relevant reading: https://www.alexirpan.com/2019/02/22/alphastar.html

They can train a discriminator network to classify actions as being generated by a human or bot. Then aim to generate actions that can't be distinguished.

The idea is to see if the bot is playing as smart as a human. To measure this, you must factor out any advantages not available to humans. This is valuable feedback when developing an intelligent agent, but that doesn't mean you can't remove those artificial handicaps once going to production with something like self-driving cars.

Edit:

Alphastar does already factor out reaction time:

“We measured how quickly it reacts to things,” Silver said. “If you measure the time between when AlphaStar perceives the game. From when it observes what’s going on, then has to process it, and then communicate what it chooses back to the game. That time is actually closer to 350ms. That’s on the slow side of human players.” https://venturebeat.com/2019/01/24/alphastar-deepmind-beats-...

> Alphastar does already factor out reaction time

That may be true in some narrow technical sense, but it was heavily disputed by the StarCraft community. If you watch the exposition games, AlphaStar has superhuman ability to micro-control its units one-by-one during critical battles, which allowed it to beat its human opponents even when its army was far inferior on paper.

Its APM and APS have been decreased since the exposition games due to feedback from pro players.

Not sure why you are getting downvoted. Upon closer inspection it was indeed perceived as contrary to what was expected from the StarCraft community. (And rightfully so)

Here is a detailed post which covers the specifics: https://www.alexirpan.com/2019/02/22/alphastar.html

I guess the thing people are interested in is a program that can autonomously play the intelligent parts of the game - the strategy and all that. I assume Starcraft is a poor choice for that, though, because of the massive importance of micro.

For instance, imagine you’re playing a chess game with a 100 ms timer and first to exhaust their timer loses. No human will win and I could create a program that could best Kasparov trivially by advancing each pawn. There’s the game and then the input layer problem.

Maybe Civilization IV ;)

There is a build time limit as well as a resource limit. Its not exactly a simple race

Precisely. Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers: https://deepmind.com/blog/alphastar-mastering-real-time-stra...

Even so, this latest version has max APM limits instated to appease pro-players. Since Alphastar is forced to perceive the state of the game through machine vision of the screen, it's reaction time is already on par with humans anyways (~350 ms for Alphastar vs. ~250 ms for humans).

Alphastar did some inhuman stalker micro that the APM debate seeks to cover, but doesn't.

Stalkers have a player operated ability to instantly move a short distance 'blink' once every 5 seconds. When in a fight, you optimally let the stalker(s) taking damage soak up as much damage as possible and then blink them backwards so they can recharge shields and continue firing from behind other stalkers. They don't stop firing, so all the work the opposing force did trying to kill a shooter, resulted in no outcome at all.

That functionality is balanced by the fact it is hard for a human to time activating the abilities of many stalkers at once in time with the damage they are taking and perform the many other actions the game requires at the same time.

Alphastar can perfectly blink back stalkers with limited apm because timing things is obviously not a problem for it, making stalkers way more value-for-money than they should be and can hold off high investment attacks more cost effectively. Ultimately sc2 is a game of economy and timing so this small change gives a massive advantage.

If I'm not mistaken, the blink micro was unrealistic because that version of Alphastar played a modified version of the game. It's "screen" covered the whole map and it had no need for a minimap. Human players can do some of the blink micro that Alphastar did, if restricted to just one screen of space.

What made its micro different was that it did it consistently from flanks on several sides of an army exceeding the boundaries of a human screen. It was also notable that it did that while macroing at home, but some macro actions during intense micro is done among better pros.

The blink micro advantage should be far reduced if Alphastar is playing the same StarCraft II installation as humans now.

> Focusing on APM is kind of a red herring anyways because during previous matches there were only a few milliseconds that it spiked above human levels, and it was usually when moving workers

If you believe Aleksi Pietikäinen -- and pretty much every one of the professional players who played against it -- the claim you're repeating here is so misleading as to be fairly considered an intentional lie on the part of the deepmind team.

For example, TLO's inflated APM are presented in that chart without comment. Specifically, without the comment that his high APM counts come from a particular game context in which holding the mouse button down (i.e. a single click with a duration) is counted by the game as thousands of APM.

https://blog.usejournal.com/an-analysis-on-how-deepminds-sta...

You are misleading people with this comment.

AlphaStar APM spiked to ridiculous inhuman levels during stalkers micro. On top of that, it controlled units that were screens apart at the same time, which is not supposed to happen.

DeepMind's refusal to aknowledge it, on top of the sketchy and misleading TLO chart didn't do them any favor.

> Since Alphastar is forced to perceive the state of the game through machine vision of the screen

Reference? I'm under the impression that the game provides several "layers of information" directly to AlphaStar, not the actual screen.

From TFA:

> Q. How does AlphaStar perceive the game?

> A. Like human players, AlphaStar perceives the game using a camera-like view. This means that AlphaStar doesn’t receive information about its opponent unless it is within the camera’s field of view, and it can only move units to locations within its view. All limits on AlphaStar’s performance were designed in consultation with pro players.

Your assumption us correct about the show matches against MaNa and TLO that many people are talking about in here. That was not the long term goal of the Alphastar team to keep it on that heavily modified version of the game. For one thing, it meant that Alphastar needed a customized version of the game that it couldn't play on the ladder. As far as an AI challenge goes, it's also really weak if the AI gets more direct access to game data than its human opponents.

I agree that trying to play video games really well is mainly about getting "cheap" experience for real-world applications, but that is exactly the reason for why they should limit APM and reaction times.

Because in the real world, you have to get a bunch of sensor data, potentially run it through it's own neural net to recognize objects, and then feed it into the decision making system. All that takes time - most likely much more time than the 1 frame it takes for the AI to make an API call.

If the AI actually played through the same interface as humans, i.e. it simply gets the rendered image as an input, and produces mouse/keyboard inputs as an output, then maybe we should disable artificial APM/reaction time limits. But as it stands now, the AI has an absolutely massive advantage simply due to using a much better interface, which it won't have in the real world.

> extremely fast reaction times are an essential and expected advantage of machines.

Not necessarily true. Yes, the machines are faster, but sometimes that is not a good thing: https://arxiv.org/pdf/1906.09765.pdf

Knowing speed runners and the like, I can imagine they will quickly find a way to determine if the player is in fact DeepMind via some sub milisecond method.

Gl hf

Reply on news.ycombinator.com