DeepMind and Blizzard to release StarCraft II as an AI research environment

More importantly, in my opinion, I'm not aware of any other AIs that have been competitive at the StarCraft problem using a Neural-Net-based implementation; from talking to PhDs in the field (but not from direct technical experience, so take this with a pinch of salt), my impression is that state-of-the-art RNNs just don't have the ability to encode the history of the game-world in a way that would enable them to be competitive. So it will require significant advances to the theoretical models to be able to compete using the sort of implementation that DeepMind will build.

Otherwise the AI is eventually just going to absolutely destroy humans with micro, which is not remotely interesting. We've known transistors were faster than fingers for ages.

How much more complexity is added by resource mgmt vs just "legal play possibilities" as now you have "this move isnt legal because you dont have the requisite resource, thus you need to depend on completing those moves first"

Are there any efforts applying deepmind to, say, commodities markets? Or something along those lines? Supply chains for complex hardware manufacturing logistics?

Robot assembly of complex parts in an assembly line?

Sure, AI's playing games is cool...

What I would like to see is an AI playing "Farm Simulator" to manage a vast swatch of farm equipment making and picking food over a vast expanse - with little human interaction.

* Auto dock

* Auto Harvest

* Auto Refuel

etc....

One AI won simply because it abused the fact it could spam monks and use the monks in a manner no human could ever achieve

They address that in the article. The context they're talking about is a bot beating a human with human-like dexterity limits in place.

Seems like they should probably tackle a game that involves strategy without micro as a stepping stone to StarCraft. Are there any games with good AIs like that already?

It doesn't matter how well you split your zerglings or move your marines, when there are tanks busting through your front door.

I can't imagine that this is where DNCs come in unless you want to needlessly complicate the matter (and create a network that is much more expensive to train).

For instance, as any protoss player knows this all too well, a Zealot placed a mere pixel away from where it should could allow a stream of zerglings in, completely throwing any game plan you had up to that point out the window and enter crisis management mode. Usually this is a "mistake", which a computer may never make, but it needs to learn the relevance of such pixel-sized mistakes, which is unlikely to have as much of an impact in a poker game or Go game as it would in StarCraft.

I think that there are two distinct aspects of the incomplete information that are significant; first, ignoring temporality, you have the fog of war, so you can't see the whole board. This is probably easier to address in a RNN, since you can play quite well as an amnesiac that just reacts to things that are currently visible. But you need to scout less if you have a memory of what's out there, so the amnesiac won't be able to play optimally.

Then there's the temporal aspect. The set of previous states of the game is not stored in the game and made available to the player, and so to play optimally you have to have a memory. This is where new techniques will be necessary.

These are separate problems, I think, so it will be interesting to see if DeepMind can make progress without reaching human performance on the second part.

Yes, but how can you stop everyone else from doing the same, if the AI part will be trivial to replicate?

This results in a very low overall competency, when it comes to winning games against highly skilled opponents.

It will be a challenge for researchers to improve performance in these areas. The problem space is very complex -- it makes solving Go look like a cake walk in comparison.

One commonality, it seems, is that players want to get better at the game. Perhaps, an AI that TEACHES the user to improve their play, and reduce errors would be an easier problem. So far we mostly do this using heuristics of gradually increasing difficulty, but it should really depend on what the player is weakest at... and what the user can improve at fastest! That is something nicely measurable, which you might use to train an AI.

Then, if/when people play against other human players they are as strong as they can be... often an FPS goal.

And a player in the top 20 percent could beat any AI that has ever been created.

AIs are really in an abysmal state for RTSes.

First person shooters can have super accurate AIs that are really frustrating to play against. Try beating the last "boss" in quake 3 on nightmare difficulty. A game like Starcraft relies much more on decision making that raw 3d math calculations and that makes making a decent AI pretty difficult without giving it advantages like extra resources or vision. That said, even though a game like Quake is easier to write an AI for, top players can still be top level AIs with superior reasoning about using the map and timing power up spawns and such.

Starcraft however has all kinds of minor inefficiencies you can build in. You could inconsistently build workers, you could occasionally supply cap, your army comps can vary in sophistication level, etc.

It is actually about making a good AI. Civ 6 or SC2, in both cases they simply "cheat" by providing multipliers to the AI in order to solve its limitations. The basic function is largely the same. Providing the AI with cheat codes is not "good" AI.

A good AI design would remove the need for such arbitrary advantages that are out of context for human players and leads to weird issues.

Where the AIs tend to be very bad is in long term thinking and overall strategy.

Blizzard's AI in Starcraft generally plays like a slightly more aggressive but novice human. It's more fun than the TA AI, but not terribly strong once you understand the game mechanics.

What wrong with a strength option? I'd include an unbeatable entry in any case.

Unfortunately, humans will find advantages that shouldn't be used in any sort of game that tries to build a believable world.

That isn't to say it's a bad idea. You'd just have to design the game with human enemies in mind. I was very interested in The Crossing [2], which incorporated that of mechanic. Unfortunately, it was canceled. :(

[1] Not a made-up problem. https://youtu.be/UfZFHdywYLA

[2] https://en.wikipedia.org/wiki/The_Crossing_(video_game)

it's necessary anyway to prevent it from having godlike micro.

https://www.youtube.com/watch?v=IKVFZ28ybQs

An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.

Unlimited APM makes AI much less interesting from the perspective of game theory.

Think of it like a robotic boxer, except instead of an android, they build two 30 foot long walls of spring-loaded boxing gloves that close in on the human boxer. Yes, the robot punches the guy a lot, so the problem seems "solved" but it isn't really.

The point is to build ai that compete with humans using human strategies. Not to cheese with borked game mechanic exploits.

Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice.

Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.

Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.