Tencent's AI plays StarCraft II, and wins over 90 percent of the time | Hacker News Reader

Tencent's AI plays StarCraft II, and wins over 90 percent of the time

eyerys.com

90 pointszeyfah79 comments

Looks like the AI is just playing against Starcraft II's AI, which makes this slightly less impressive, as I first thought it was against human players.

The SC2 AI on higher difficulties does cheat though (extra resources) so it does make it slightly impressive, but still a far cry from human opponents as SC2's AI is pretty easy to beat (it would be around Gold in ladder I think)

If it's just the built-in AIs, you're right that it's much less impressive. They should be winning 100% of the time.

For example, I don't know if they've fixed this, but you used to be able to beat the AIs 1v4 on the highest difficulty. There was a bug where the AIs would surrender individually even though they were on a team. So with an all-out rush you could get two of them to surrender almost immediately by running 2-3 zealots through their mineral lines. Getting the third one to surrender required a bit more fighting, and then the fourth was more of a normal game.

The AIs also used to keep their entire army in one big death ball, and if there were 3-4 fighting units in their base they would send the army home to defend. Even if their army was 10x bigger than yours and they'd win the base race. So you could hop reapers into and out of their bases to prevent them from attacking your base indefinitely (because they kept running home). Also you were doing pretty serious damage with the reapers while they ran back.

Another exploit was to start building a cheap building in their base area at the start, somewhat far away from the mining line. They would send a ridiculous number of workers to attack your building. So they'd lose significantly more money to lack-of-mining than you did for building and cancelling a pylon or whatever.

> Another exploit was to start building a cheap building in their base area at the start, somewhat far away from the mining line. They would send a ridiculous number of workers to attack your building. So they'd lose significantly more money to lack-of-mining than you did for building and cancelling a pylon or whatever.

Similar to this none of the inbuilt AI's can deal with a slow cannon rush. They will hesitate for ages then rush 5 finished cannons.

The later Ai's do a little scv tanking and kiting but nothing near enough, they don't even focus down the probes.

If you float an overlord just out of range, sometimes their entire army will just sit their watching it. I've accidentally won many vs AI games this way. There also some 1 base all-in's that I've never had the AI defend.

> If it's just the built-in AIs, you're right that it's much less impressive. They should be winning 100% of the time.

Absolutely. Unless the win-loss ratio includes all games while the AI "learns" to play the game. It might have lost most of its early games, but now wins 100% of the time.

I suspect it wouldn't take DeepMind long to master starcraft II and beat starcraft's AI 100% of the time.

It also reacts very badly to liberators zones, compared to like tank zones.

Vs 4 insane AIs, I floated my command center to an island with 2 mineral patches and produced mass battlecruisers =D

It would be bronze. The built in AI is much worse than you think.

Bronze is now the bottom 4 percentile. I was around the level of Very Hard while 2600 NAMMR (High silver)

This is not true. Maybe when the game first came out, but average skill level has decreased greatly.

This is counter to the current perception of professional players, which is that world champions from ten years ago played the way that unexceptional ladder players play today. I think the average skill level has been increasing monotonically over time.

Fine, overall skill has increased. But given the vast amount of information and strategy guides, it is much easier to get to a diamond level now than it was before.

That idea doesn't seem coherent to me -- Diamond is just another way to say "top 60-90th percentile". The vast amount of information is available to everyone equally, so it doesn't explain why some people are above average players. Players in every league have improved. There's no static "diamond level" way of playing that's been unchanged over ten years to compare against.

I’ve played since the release date thru now with gaps of time not playing... from my experience it’s unequivocally clear that the skill level is much much higher than when the game first came out. Like not even close.

That's true. Watching the first WCS championship is extremely cringy even for a normal-skilled player today. It's cool to see how the meta has improved in recent years !

You mean average skill has increased?

If true that would make it pretty much alone along long-running competitive games.

Can confirm, beat all of the AI levels reliably before moving to ladder, and placed in bronze and stayed there for a few months.

> it would be around Gold in ladder I think

In the article, they say it plays around Platinium / low Diamond, when they had it play on the ladder.

Though also: "It is worth noting that although TStarBot1 can successfully learn and acquire strategies to defeat all the builtin AIs and TStarBot2, it lacks strategy diversity in order to consistently beat human players. In the aforementioned test with human players, TStarBot1 will be unable to win once the human player starts to know TStarBot1’s preference for Zergling Rush."

For anyone who actually plays SC2, or any real time game where human speed is a limitation, it should not be surprising that AI would crush human opponents.

This video demonstrates why: https://m.youtube.com/watch?v=IKVFZ28ybQs

AI has crushed humans in Go, the most complicated (widely played) board game. I'll be impressed again by AI when it beats us in physical sport.

What would be interesting is if they limited the AI's ability to that of humans. Ie - simulate how fast a human could potentially group select and point-click, etc. Now THAT would be interesting.

Until then, yes as you put it, the AI would crush humans and unfairly so.

Build a set of robotic hands with human nervous system-like control inaccuracies and delays. Put it on a mouse and keyboard. Point a camera at the screen (tuned to match human visual acuity and eye movement speed and focal area), put headphones on a mic, let the AI play the way humans do.

Why doesn't human speed/physical strength lose out in physical sport controlled by AI as well? Isn't the analog of letting an AI take actions in SC2 as fast as it can make decisions just letting an AI control some robotic body arbitrarily better at tackling and running and whatever other relevant physical attributes than humans are?

The article is talking about their AI crushing other AI's, not humans. There aren't any AI's right now that can beat high level human players, even with superhuman reflexes and control.

> AI has crushed humans in Go, the most complicated (widely played) board game.

In terms of ruleset, Starcraft is at least a few orders of magnitude more complicated than Go. It doesn't feel more complicated to humans, really, but the sheer number of variables at play is vastly higher.

Brood War has a significantly higher branching factor than Go: each of ~50 units can receive 3 orders with arbitrary targets (the untargetted ones hardly matter for branching). The map is 4096 x 4096 pixels, though some of those are functionally identical so say 256 (the number of angles a unit can face). So branching factor around 38400. The game runs at 24 fps, and the average game is 20 minutes long, for 480 turns. 38400^480 is a number with 2201 digits.

Starcraft II runs at a higher frame rate and uses higher resolution maps, so these numbers are even more ridiculous there.

> game runs at 24 fps, and the average game is 20 minutes long, for 480 turns.

Messed up your conversion there, fps is seconds and you had game length in minutes, so need to multiply by 60. So that's 28,800 "turns".

Starcraft 2 is not "orders of magnitude more complicated than Go". It's far less complicated in terms of options, and far more complicated in terms of factors like speed/reaction time that don't exist in Go. Those things are difficult to compare at all, but certainly do not qualify as "orders of magnitude", which is measurable.

> It's far less complicated in terms of options

Could you explain how you figure that in more detail?

I mean, to my untrained eye, it sounds like it wouldn't be so, since in every time slice of the game (equiv. of a turn in Go?) you can have hundreds of points of control - hundreds of levers to choose to pull - and of course that's in every frame (or whatever interval the UI actually allows you input).

It absolutely is, which is partly why there aren't any good AI's for it yet, even though we have ones that can crush the best players in the world at Go.

Go's game state, for example, is very small. You can pretty much represent it with a 19x19 array of 2-bit variables, because each space only has three possible states. That's only 722 bits. Maybe a few more to track whose turn it is and how many pieces each player has remaining.

In contrast, Starcraft can have hundreds of units in play from dozens of types, each of which usually has at least a position value (x and y) and a health value, and commonly has other things like energy value and cooldown values. And each of these you'd need at least a 16-bit int. And that doesn't even get into the enormously larger possible action space for each "turn".

Anyone who's played starcraft competitively knows how much infinite potential there is to micro control, much more compared to Dota2, and that even the best players can't reach that level either by a very large margin.

Dota2 is not the same way - some of the best players perform micro-control on par with AI. This makes AI in DoTA much more impressive

To me, the way you've explained it makes DOTA less impressive as a game.

Starcraft focuses more on mostly one critical pivot micro battle, and many compounded macro processes leading up to that point.

Dota II focuses more on compounded micro decisions throughout the entire game for sometimes 60 mins+. Dota II revolves around using your one hero (with the exception of meepo), and teamwork much more heavily. The amount of different scenarios in DoTA II is significantly higher because of all the different heros, skills, levels, and combinations thereof.

Starcraft has very predictable scenarios though. There is usually less than 10 meta builds at any given competitive nerf/post patch balance. Dota II is hundreds if not thousands, per hero, per team composition, etc. So AI is way more impressive in Dota II

>There is usually less than 10 meta builds at any given competitive nerf/post patch balance. Dota II is hundreds if not thousands, per hero, per team composition, etc.

There are hundreds if not thousands of builds for each hero?

Go is a perfect information game. RTS are not.

And the micro advantage is not all that relevant since APM and latency restrictions can be imposed on the AI to force them to play under conditions comparable to human players.

Depends on whether or not the AI can see through the fog of war. In some RTS the "insane" level AI has full knowledge of the map. But yes, if it has limited info, like human players, the strategy is much different and much harder for an AI to intuit compared to chess or Go. Unless the winning strategy in the game allows for constant scouting, but is unlikely due to the waste of resources on that unit

Deepmind states in their released papers that they limit the actions per minute for the AI so it doesn't just win via speed. Their current approach is a bit simplistic and does not really reflect how humans perform actions but they want to keep it somewhat fair on the "mechanical" aspect of the game. OpenAIs Dota Agent also has similar restrictions although it has shown super human reactions in some cases. This should be tweakable in such a way that the Agent is on a human level and needs to actually be better at the decision making.

This comes up every time Starcraft AI is discussed. It's really frustrating. Why immediately discount the incredible work AI researchers are doing by instantly assuming they are using cheap tactics like shown in your video?

When a Starcraft AI is ready, it will be incredibly obvious if it's using brand new builds, strategies, and tactics (which pros will immediately copy for the next $500,000 tournament) or if it's "just" using 10,000 APM to perfectly dodge attacks in a way that a human never could.

I'm guessing Google and Tencent are going for the former instead of the latter.

You could fix this by capping the apm of the AI to 500

I feel like "useful" APM and "raw" APM are very different for humans than it would be for AI. You can technically get an AI to "optimize" their actions to the minimum required for the end result, but humans tend to frantically waste actions during a game.

I would cap it at 300apm, thats how high most professional koreans are IIRC. But that 300 apm is 90% redundant mouseclicks though

I'd really like to see Starcraft AI do counter builds and fake out human opponents with falsified scouting information. I can see AI eventually becoming good at information manipulation and taking advantage of human psychological tendencies.

Machines winning at physical sports has already happened. It is even less impressive than any AI winning at real time games. A race car is faster than a human runner. A speedboat is faster than a swommer. A tank would be an untackleable running back. A rail gun Tennis serve would be unreturnable (etc)

Yes but probably parent meant a fully automated opponent, e.g. not car + human driver but an autonomous car. That probably beats any human in Olympic running races even now. Robot tennis player, not yet.

Also a factor is that the game is balanced for human inputs. As many crazy micro trick videoes demonstrate by doing things like optimal gapping zerglings against siege tanks.

If players could do that reliably we would start seeing major unit changes. It already happened to the original void rays who would do things like attack themselves to maintain charge to keep massed annihilation capability.

Players can’t do that mostly because it would be impossible to correctly determine which zergling was targeted, this bot has access to perfect game state information from the sc2 editor api and it looks like it is using it to me.

However, a bot could use just the same information a player could theoretically have access to but humans can’t pay attention to it all at once- that would be a real advantage. This one is basically cheating extra information though.

I’m pretty sure that program in the video has access to internal game state to know which zergling is targeted, because it splits them when the tanks are out of sight, and always picks the right zergling. A real bot wouldn’t have access to this state, and the one in the article doesn’t. Marines vs zerglings+banelings would probably be a better micro example showing theoretical bot perfect-but-fair micro.

In any case, this can be solved by capping the actions per minute near Pro human levels, somewhere in the 300-500 range.

There are still incredible strategy problems to be solved that cannot be overcome with perfect micro. Figuring out a build alone using RL (which is what I think this paper is doing) is still a huge step.

That's not just super human reflexes it's using information no player connecting through the normal interface has.

If you actually play SC2 (or more significantly, if you follow the professional competitive SC2 scene), you should definitely expect AIs to be a looong way away from competing with professional humans. A pro human with an AI assistant (either to micro for them, like in that video, or macro for them) would obviously be able to beat another pro human on their own, but an AI by itself won't stand a chance with current AI.

I would watch an AI-assisted human tournament, especially where each team is responsible for designing their own AI

It would certainly be interesting. In professional SC2 competitions even simple input macros like fast-repeating clicks or keystrokes are banned. It would likely be a massive advantage.

I play SC2 as well as work on AI/ML during my day job. The state of AI is extremely hyped by the media. The most sophisticated neural networks I have seen so far are recommender systems. The algorithm that predicts which next movie on Netflix you might be interested in based on your history of completed and abandoned movies. However, for all such systems there is a precise way to measure the effectiveness of the AI and improve it by doing A/B testing etc. Main problem with SC2 and AI is there isn't a good heuristic to measure your progress at any point of time. Eg: If player 1 has 25 reapers and player 2 has 8 void rays who is ahead? Well the void rays could destroy all barracks and win or the Terran player might make a few marines and missile turrets to counter the void rays while attacking probes with reapers. Basically, if you pause an SC2 game at any point of time(barring last 2 minutes) you cannot predict the winner with 100% accuracy. This is a big problem for designing an effective AI for SC2.

It would greatly depend on positioning and chance of person winning fluctuates depending on the number of optimal moves available, similar to GO but more complex since there is so many scenarios.

Also you have to consider whether voidrays have speed upgrades, same with repears. Terran can play with fog of war too and lift off, usually terran has an advantage here

The article published: https://arxiv.org/pdf/1809.07193.

> we propose to model the action structure by hand-tuned rules. By doing so, the available actions are reduced to a tractable number, which turns out to be easier for designing our decision-making system.

It seems they are not using the hard-core Blizzard/Deep-Mind interface (pixel-based), but instead using an actions-based one, similar to Starcraft I bots and Open AI's Dota bot. Still pretty interesting though.

For those interested, there is an excellent Youtube playlist by sentdex that covers building a Python AI to play StarCraft II. He's up to 17 videos so far and it's a fun way to learn/practice neural networks.

https://www.youtube.com/playlist?list=PLQVvvaa0QuDcT3tPehHdi...

Another link: https://thenextweb.com/artificial-intelligence/2018/09/20/te...

While others are talking about how "easy" the built in AI that can crush me is (I'm pretty bad), the part I found interesting is that they taught it to "see" the map from the data. Nothing new in the field, but interesting to me.

Part of me wonders if the techniques applied can be easily used on non-visual data. AI that "sees" what we cannot. (I realize we aren't talking any kind of actual AI here, but sometimes it is nice to just wonder... )

Isn't deepmind also building an AI for Starcraft? Deepmind vs Tencent would be a great playoff. I have a feeling deepmind might dominate tho.

Deepmind collaborated with Blizzard to create the SC2 AI API: https://deepmind.com/blog/deepmind-and-blizzard-open-starcra...

With OpenAI's Dota playing AI making the rounds over major media and serving as a major advertisement for the game itself, why would Tencent develop and publish an AI of a game they don't own (Starcraft) as opposed to one they do own (League of Legends)?

Seems like a such a trivially better decision

They don't own League of Legends; they have a significant interest in the company which created it. League also doesn't have an API while Starcraft 2 does.

https://en.wikipedia.org/wiki/Riot_Games#History

> In February 2011, Tencent paid $400 million for a 93 percent stake in Riot Games.[2][8] Tencent bought the remaining 7 percent in December 16, 2015; the price was not disclosed.

I don't know about you, but having 100% of the shares seems like a quite a bit more than "significant interest" in the company. Some would say that this "significant interest" qualifies as ownership :)

If I recall correctly, the ownership structure is set up in a way such that Riot has creative license over the game - when Tencent wanted Riot to build a mobile version of League, for example, Riot refused: https://www.theinformation.com/articles/tensions-flare-behin...

From the article:

> Two years later, after watching the game take off in the U.S., Tencent raised its stake to 94% at a price valuing Riot at about $400 million. The remainder was held by the founders and employees, who wanted to keep at least some equity.

> That deal would prove pivotal in what was to come. To persuade Messrs. Beck and Merrill to sell most of the equity, Tencent’s international chief David Wallerstein negotiated an agreement that ensured Riot could operate almost like a separate company. Tencent took two seats on Riot’s five-member board, while Riot’s founders took two others. The fifth seat was empty, a structure which remains in place.

So yes, technically Tencent owns Riot, but they don't control Riot. I guess from a semantic viewpoint I was incorrect in that they "own" League of Legends, but as far as Riot is concerned Tencent doesn't have any control over its development, including building an API to support AI/ML research.

Maybe because then they couldn't be accused of knowing the inner workings of the game? I think they picked Starcraft because they can truly handle it like a black box.

OpenAI gets very low level access to the DOTA source code and bot API, there is nothing hidden there but it facilitates integration.

Tencent just owns a majority stake in Riot they didnt develop LoL and have very little to do with the actual game more likely they went with SC2 because it’s easier to integrate with as it has a bot API and it looks less like they are copying OpenAI and making another MOBA AI.

SC2 and DOTA/LoL are also fundamentally different games with SC2 not being cooperative and presenting a different set of problems and solutions than MOBAs.

After Deep Mind beat Go (and before), there was a lot of Starcraft being "the next thing" for AI. That's why a lot of research institutions (including Deep Mind and Facebook) are working on that.

In fact, it was always a bit suspicious that Open AI chose to work on a different game, which may or may not be as hard. Having multiple groups working on the same problem makes it easier to understand the progress made.

I suspect Dota 2 is a lot easier than Starcraft. I watched a video of that AI, and it was mostly tactical optimizations ("micro" in Starcraft-speak). The AI would position its unit just right, aim perfectly, optimize for maximum damage. The stuff machines are best at.

Dota 2 is tactical, so having perfect micro is enough to win at least simple matches. Starcraft 2 is a different story.

Slow humans is not as much a bottleneck in Dota as it is in Starcraft. It's easier to "brute force" Starcraft by sheer APM, which is easy for AI to do.

Nobody would find that interesting though. Just like the Dota not, and Deep Mind's Quake bot, accuracy and APM would of course be capped to human levels.

Because Starcraft is 1v1 and LoL is 5v5. Poker AIs also focus on 1v1 as opposed to full ring games. It's easier.

Also because Starcraft has three races, i. e. 9 matchups. But if you pick one race for your AI, you only have three matchups to worry about. LoL on the other hand has about 500 champions (give or take), whose strengths and weaknesses are vastly dependent on the current patch. LoL is less about skill or strategy, and more about picking the champion of the week.

Starcraft works because they have such a small number of matchups that balancing them is possible. Certainly not easy, and players keep figuring out ways to evolve the meta game. Maybe an AI could even find interesting new strategies. But LoL has so many matchups that their balancing team has an absolutely impossible task of trying to catch up. It's so complex that they can't just think about in theory, but they have to look at win-rate statistics, over different skill ranges. At this point it's more a problem of data science. And the number of champions keeps growing, because that's how they make money.

> LoL on the other hand has about 500 champions (give or take)

LoL has 141 champions. Still significantly higher than Dota's 116.

A common note I hear when people discuss open AI playing dota is that it uses pre-made matchups, which reduces the number of matchups considerably.

Would it be that difficult to have the AI play the picking stage? Compared to the complexity of the decisions you have to make during the game, the picks are straightforward - especially if you have performance data on matchups.

The number of practice games the AI would have to play to learn to handle all the heroes goes up drastically, but that's only a matter of time.

I'm pretty sure OpenAI only plays the single simplest hero, mechanically. The other 115 would be varying degrees of "more difficult" to play competently.

Also, it only played mirror matches. If it had to play any hero against any hero, that's 3.393109e+190 possible matchups.

They've had some updates since then: https://blog.openai.com/openai-five-benchmark/

(Plus two shorter / more recent status updates)

I don't think you need to learn all possible matchups to play any matchup, as a human would. You'd need to just learn how to play/counter each hero, plus some 2 or 3 hero combos (not all permutations), so on the order of a hundreds?

They had drafting working just fine in a previous version of the AI. (The one that played against semi-pros a month or so before TI.)

One theory I heard was that the pre-made matchups at TI were designed to prevent the human players from getting out-drafted. (The hero pool was pretty limited, which could result in a different meta from what the human players were used to.) They wanted to make the matches purely a test of in-game skill, not about the metagame. As far as I know though, the OpenAI team hasn't explicitly confirmed that.

  > Because Starcraft is 1v1 and LoL is 5v5
  Starcraft can also be played 5v5
  > Also because Starcraft has three races, i. e. 9 matchups
  But each race has 20 different units. The units are more 
  similar to LOL champions than the races. There are also a 
  tonne of different buildings and upgrades to choose from.
  LOL is very similar to DOTA and given that OpenAI already 
  beat *human* players in DOTA, LOL is not that farfetched. The 
  current work, although impressive, still cannot beat *human* 
  players in SC2. The complexity of designing AI for a game 
  depends heavily on the branching factor of decision in the 
  game. SC2 has a much higher branching factor than LOL or any 
  other games played by n00bs.

Has there been any crossover success with gaming AI to other problem spaces? It seems like they're just highly optimized for a particular game and can't necessarily replicate that success outside of gaming or even in other types of games.

It was never meant to train on Starcraft to dominate Solitaire or Mario, or to figure out a more efficient way to do laundry. There's no money in that. The particular interest in Starcraft (an iconic wargame) is meant to lend itself to the military applications of logistics and troop placement.

At some point AI will be sufficiently advanced to understand what a grunt on a map looks like in any context, so we'll be able to swap out the Space Marines with real ones. Whoever can autonomously dominate a battery of RTSes has the potential to autonomously conduct conventional warfare.

Tencent owns PubG mobile. I wonder if they're training bots from actual players from that game and any other ones they own.

I know bots inhabit that game, but they're not very goo.d

Site seems to be down - alt link: https://is.gd/gj6Y9M however, as this article states, the bots learned to do a zerg rush not more which is really not impressive. I would like to see ingame POV footage to actually get a grasp of how these bots perform

Reply on news.ycombinator.com