Humanity Has Hope: Pro-Gamers Win 1 of 11 StarCraft Matches Against DeepMind AI | Hacker News Reader

Humanity Has Hope: Pro-Gamers Win 1 of 11 StarCraft Matches Against DeepMind AI

fortune.com

27 pointslawrenceyan36 comments

Teach an AI how to take and keep control of a battlefield. What could go wrong?

That's easy: not doing it.

Does the AI take a video input or is the AI still operating through a direct API into the game? If so, it's a bit like cheating. Pure data access and no visual noise to filter through is a huge advantage for it but still impressive.

I believe the AI still has a direct API to the game and in the 10 matches it won it was able to effectively "see" and issue commands to the entire map all the time.

In the last match they used a newer interface where the AI had the same restricted viewport as a human and had to decide where to move its camera to both see what's happening and make actions. The player and commentators both noticed that this resulted in more human like play, and the AI seemed less able to micro multiple armies stretching beyond the viewport like it had in earlier matches.

The deepmind people said the restricted viewport did slow down training, but in the end it was able to get to the same skill level as the previous ones in the end.

If the ai plays equally well with restricted viewport why were the majority of the games played with nonrestrictive? It seems reasonable to assume that deep mind will eventually best humans at sc2 but seems like Google wanted a nice headline and rushed it imo.

Because they only added it recently. About a week before the livestream. The other games were recorded last year.

I agree, it's an apples to oranges comparison. The human has 10 fingers, a max amount of input keys it can reach with 1 hand, input lag between eyesight/motor-reaction/mouse-movement from point a-to-b. If we were trying to measure intelligence, we'd need a human with some more hands/fingers and input devices or an AI that has similar input limitations.

In the videos, there's the Actions Per Minute metrics for both the AI and human players. AI average should be around 100, the human is much higher (300 or so) - although it's still apples to oranges, at least the AI isn't going full throttle on the APMs.

they would have won more if they played it more when it was only limited to single view mode instead of playing off the entire map where it was able to complete surround and micro a bunch of stalkers and defeat immortals. Also, they should have gotten sos to play against alphastar instead of mana

And they would have won none at all if deepmind had worked on the problem for one more year before showcasing it, even with this limitation. I don’t think it really matters.

Not having to move the camera around is massive. High level starcraft has been likened to something like poker by many pros. Except, instead of getting a steady trickle of information in terms of the cards being put down you have control over what information you gather. But the information is spread out across space and time and the viewport only gives you a small slice. And the human eye only lets you look at a small portion of the screen as well. If you check your minerals, gas, and supply, you cannot be checking your minimap at the same time. Alpha starcraft can look at the minimap constantly while knowing the position and health of every single unit not in the fog of war.

For human players maintaining this map awareness is definitely the hardest part of starcraft. The micro isn't really that hard, you can train on special custom maps and get most of the tricks down pretty quickly. The build orders and meta level strategies aren't very hard to wrap your head around either, the possibility space is actually pretty small here.

What is hard is constantly task switching, not hyper focusing on a couple of units, knowing what you should be focusing on at any given point in time... Well, those are all pretty much the same, it is, "how should I spend my very limited attention".

If your opponent has infinite attention resources some strategies that work against humans dont really make sense. Drops for instance are balanced around the idea that if the opponent recognizes that they are being dropped they will be able to pretty easily defend against it. Against a computer that will instantly see the drop the millisecond it comes out of the fog of war... Well, you just gave away the hard part for free. A human player would either have to a) get lucky and happen to glance at the minimap in the small window to counter the drop. or b) anticipate the drop and make the conscious choice to spend precious attention rescources to catch it.

> Not having to move the camera around is massive.

If it's so massive, why did their camera-limited agent have near-parity against their non-camera-limited agents?

I thought the camera limited match was the only one it lost?

Once alphastar was faced with a scenario that couldn't be brute forced with perfect micro it pretty much stopped working.

Internally it played thousands of training games and performed en par to the non-limited agents.[0]

A far bigger factor is that this new agent simply trained for fewer days because they only implemented it recently and it was hot out of the oven. Give it another week of training and it would probably be a hard nut to crack again.

That said, the matches are played in a manner that gives the humans little time to look for idiosyncratic AI behavior that's exploitable. If they played several dozens of matches against the same set of 5 agents again and again they would probably figure out some cheesy strategy that the AI doesn't know to defend against.

We're not yet at the point where the AI can adapt to counter-adaptions in realtime.

[0] https://storage.googleapis.com/deepmind-live-cms/images/SCII...

The Dota bot was easy to exploit after some game against it to figure it out for example.

Overfitting maybe.

The show off game: https://www.youtube.com/watch?v=cUTMhmVh1qs&t=2h31m22s

The bot is getting it's main force lured back to it's base by hit and runs for it's resource producers. Repeating the same mistake again and again. Later it acts confused and rather have it's main base wrecked than having a high risk fight with it's main force vs the players main force, even though it would auto-lose not doing so.

I've got a feeling these bots have a huge API advantage in "mechanical" skills versus humans. Eg. it has to be way faster to retrieve the "x ability used by y" event than seeing an animation unfold on the screen, leaking intention to bots earlier then human players.

Can you provide an explanation of what you said for those of us who don't know much about Starcraft or competitive RTS games?

I find what you said interesting but I lack the context to understand it.

A lot of these are glorified and complicated "rock paper scissors" games. Each race has a set of units, some air, some land. Most can attack anything but others are specialized and can only attack land or air. Some have really long range, but other hindrances to balance them.

Yes this is a mass simplilfication of the game. There is a lot to resource management and production capacity to train/build/grow/call in more troops.

I cannot speak to the competitive side of it.

The computers see and can click the whole battle field and the humans have a view port at a specific area. Star Craft is very micro oriented (give orders in real time to units rather than just clicking attack and wait), so the computers have a huge, unfair, advantage in being able to see everything at once and micro everywhere at once and react faster on emerging enemy units, since they are always looking everywhere.

Zooming out for a human player would be considered cheating. It would be easier to see hit and run attacks and react in time etc.

AlphaStar played 11 games. In 10 of those games, it played in a mode where it didn't have to manage the camera, so it didn't have to context-switch between things going on in different parts of the map, the way a human would. It won all 10 of those games, using a strategy which a human would not have been able to pull off. In the 11th game, it played with the same camera restrictions a human would have. It lost that game.

The game where Mana won seemed to be primarily because he discovered an exploitable maneuver against the AI.

Yes, AlphaStar was already up 15 workers -- "build a Phoenix" would have completely shut down the prism and is apparently something that particular agent never learned how to do properly.

Camera management isn't part of that equation.

Though it is fair to conjecture that Mana may have won the earlier Blink Stalker vs Immortal game, if camera management were required.

Depends. If camera management soaked up a large share of the training capacity of the network, then that might explain why it failed to learn how to handle drop-harrassment.

It is also time to move on to real world environments. Games are cool, but the network is essentially learning some thousand lines of C++. Reinforcement learning needs to prove itself in real-world tasks.

I'm a layman, but to my knowledge navigating a digital environment and a real one are the same minus some steps of the process. A self-driving car recreates a digital reality via sensors with as little delay and fidelity loss as possible, then navigates a digital car within that rapidly constructed virtual reality. It then signals back to the real car to navigate the real car exactly how it would navigate the digital one given its immediate digital environment. Since many of the hard problems lie with the navigation in the digital space, removing the sensors-to-digital part of it by training with video games still nets a lot of valuable learnings that can be brought right back into real-world applications. Also, learning in a fully digital environment allows this part of the learning to be done without the spatial/time constraints of reality.

games are simulations of idealized processes using generally a small number of equations. a sufficient neural network should be able to learn them but the real world has orders of magnitude higher complexity. Something similar happens with robots trained with physics engines.

One step at a time sounds like a better strategy.

Watch this and learn a bit more about why they're doing Starcraft AI. The Deepmind guys first appear around 32:00.

https://www.youtube.com/watch?v=cUTMhmVh1qs

The problem with real world tasks is you can’t really train on them.. you train on a simulation (because its much, much faster/cheaper to run [and debug], thus faster to train) and deploy/test in the real world.

So simulation is always a part of it, regardless of target execution environment. In which case, games make a lot of sense, since they’re already complex simulations (thus useful for learning how to train AI’s, and what parts of the simulation are worth having), make good PR, can be tested against quickly, and in the case of competitive games, have a set of “experts” that already study what the human-powered collective AI learned over time, and thus it comes with a set of known expectations of how the AI might fail, and what we expect it to learn.

So tldr; simulation vs real world is a red herring. It’s the same learning process, just different execution environments.

This kind of garbage needs to be downvoted...

This was a failure on Deep Mind's part, not a success. The bot is not given remotely the same context as a person. It will lose in that context.

You can make a fighting game AI without deep mind that will parry every attack and punish with a perfect combo and always beat humans because of 0 reaction time and 0 execution flaws. Big whoop.

This is irritating clickbait.

It really is. It's stupid people building stupider machines and trying to pass it off as 'intelligence' so they can make $$$.

These things are starting to get annoying. What's the plan here? Make a game AI MVP, make sure it wins against pros in a PR event, make lofty statements how promising the new AI revolution is & make sure to make vague connections to AGI and that the human brain is soon obsolete, and finally wait for the AI hype to pour funding at you to do some bullshit project with the primary purpose to make some corp/exec look modern?

With all due respect, I think your perspective is both limited and narrow minded. Games are how we as humans develop in adolescence. They are incredibly powerful tools in helping us through the process of learning how to learn.

All the model architectures and machine learning techniques that have been developed through the playing of these games, exponentially increasing in difficulty from Go to Starcraft, can be directly applied to real life tasks.

What is life anyways if not just a superset of sets of games to be played? Optimizing the output of a protein structure designed for chemical catalysts, teaching a car how to get from point A to point B while navigating a rule based obstacle emplaced environment, etc, etc. All merely simple games, and yet filled with pretty much infinite possibility.

Underestimate the power and potential of machines learning to play at your own folly.

That entire reply is a great example of "vague connections to AGI".

I hope this AI hype cycle ends soon and funding gets diverted from tech to actual science in the medical field. But I guess that's unlikely. I see no obvious evidence for AI/ML etc to be less of a speculation hype than the already deflating self-driving car hype.

I don't doubt that ML will have its practical application given the amount of effort that is currently put in to it, but I do think it is wildly inflated. Not to mention that the spectrum of what kinds of software that counts as AI seem to get wider for every week.

That the AI hype is more or less entirely confined within an already hyped tech industry doesn't seem to me like a coincidence. An AI revolution hellbent on figuring out how intelligence works would surely include other fields than just tech?

https://deepmind.com/blog/alphafold/

https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp...

Is there a video of the game where AI lost?

It was live streamed, you can watch it here: https://www.youtube.com/watch?v=cUTMhmVh1qs&t=2h31m22s

Tthe first 2 and a half hours goes over the other 10 matches, which weren't played live on stream.

Reply on news.ycombinator.com