Any predictions for how long it will take for an AI to win against the world's best player?
I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without being able to see what they're doing, and more importantly intuiting what your opponent isn't ready for.
The biggest difference, though, is the "RT" in "RTS"-- real time. This isn't turn-based anymore, where at a given moment you have a single choice to make, a single piece to move as in Chess and Go, and can then wait for the singular and visible reaction your opponent makes before making your next choice.
My understanding it that the moves a program like AlphaGo makes are not interconnected-- it picks each move individually as an ideal move for that board state. It could take over halfway through the game for someone else and would make the same move that it would have made at that point if it had been in control the whole time and arrived at that board state on its own.
But that doesn't work in a real-time game, since you and your opponent are now moving simultaneously and the "board" is never static. Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last. There is no "one" move for a given state.
Another facet of real-time play is the idea of distraction. It's very important in RTS's to keep your opponent distracted, to disrupt their plans and their focus, by coming from unexpected directions at unexpected times, sometimes concurrently with other operations against them. This can't happen in Chess or Go, where the demands on your focus are far less urgent and two things can't happen at once in a literal sense. Can an AI agent learn to appreciate the power of distraction? Can it learn to intuit what will be most disruptive to a human, and what won't be disruptive at all? How can you teach a computer to learn to be annoying?
I will say, of course, that nobody saw AlphaGo coming. And I hope it's the same with RTS's. That would be so exciting. I would love to see an AI blow us away with previously unthought-of strategies. That would be the coolest thing ever. So I hope it happens. But I'd be astonished. RTS is just such a whole new level of thinking for AIs.
If you play using the UserPatch on Voobly, where the serious custom AIs are written, you can play vs. Barbarian. It is _very_ good, if you're not a semi-competitive player it will certainly beat you.
The upcoming UserPatch 1.5 will add even more features, so Barbarian and other custom AIs will become stronger.
To be clear, AoE2 AIs are all rules-based and written by pro players themselves, which is quite different from what DeepMind is trying to do.
http://spectrum.ieee.org/automaton/robotics/artificial-intel...
As soon as you can build a probability distribution over possible states, you can use Monte Carlo like methods.
I haven't played AOE2 so I don't know if the mechanics are similar enough to translate, but my goal for my Starcraft bot is to do precisely this. If you can enumerate the possible builds (what's available when) and assess the matchups between builds, you can make this happen using some intuitive expansions on adversarial search.
> Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last.
Recomputing the entire plan from the current state works in RTS too, but only if your decision-making takes every already-in-motion thing into account and has no internal discrepancies. That's a pretty big if; this sort of weakness accounts for a lot of bot weakness currently. Units spinning around due to slight changes in perceived state cause lots of wasted resources.
> Can an AI agent learn to appreciate the power of distraction?
Despite multitasking theoretically being one of the strengths of an AI, a lot of the current field can't handle more than one military situation at a time. In this year's SSCAIT a lot of bots completely fell apart when confronted with one of the top bots (Bereaver) doing reaver drops.
I'm not sure a bot can meaningfully learn distraction, but I'm not sure it's necessary - attacking on simultaneous fronts is optimal anyway. The army can only be so many places at once.
An example of a non-trigger is knowing that if I haven't seen a certain unit at time X, I know I'm safe to do Y. It is acting upon the information that something didn't happen.
To expand: I saw my opponent starting two gases at my 21 supply scout. When I scouted again at 47 supply, I saw no gas heavy units, so I can deduce the gas was used for better technology. This will allow me the opportunity to increase my worker count by Z before building army, or I could try and kill my opponent right there for his technological greed.
With SC2, no AI even comes close to beating even a silver level player, so even a 5 year timeline seems really soon. Let's see if DeepMind can beat it!
What's your totally unscientific guess, Gwern?
\ "Learning model-based planning from scratch" https://arxiv.org/abs/1707.06170 , Pascanu et al 2017; "Imagination-Augmented Agents for Deep Reinforcement Learning" https://arxiv.org/abs/1707.06203 , Weber et al 2017 (blog: https://deepmind.com/blog/agents-imagine-and-plan/ "Agents that imagine and plan"); "Path Integral Networks: End-to-End Differentiable Optimal Control" https://arxiv.org/abs/1706.09597 , Okada et al 2017; "Value Prediction Network" https://arxiv.org/abs/1707.03497 , Oh et al 2017; "Prediction and Control with Temporal Segment Models" https://arxiv.org/abs/1703.04070 , Mishra et al 2017
Yeah, I suspect you're right. Eliezer was alluding to this with the AlphaGo victory as well:
> ... Human neural intelligence is not that complicated and current algorithms are touching on keystone, foundational aspects of it. https://www.facebook.com/yudkowsky/posts/10153914357214228?p...
I can't decide if I would be bummed or excited if that turns out to be the case. On the one hand, we'd be that much closer to AGI. On the other, we'd be continuing down the path of brute-forcing intelligence, rather than depending on those elegant, serendipitous breakthroughs that much of human progress has been built on.
That's brute forcing as well. One such elegant idea comes every million(billion?) people. Random people would just output random ideas.
[1] http://spectrum.ieee.org/automaton/robotics/artificial-intel...
I don't want to be overly semantic or PC on HN, but just saying the GP may be female, judging by their name on profile. Being misgendered could be very offputting and discourages participation, so you may have wanted to say "this person" even if it doesn't sound as offhand as you'd have liked it to come across.
> I don't want to be overly semantic or PC on HN
Proceeds to be overly semantic or PC on HN...
- Macro: resource prioritization (army vs expansions vs upgrades), scouting to understand the opponent's macro choices, choosing the right posture in response (defensive, harassment or offensive), and resource optimization (not getting supply blocked, scaling production with income, increasing income at the maximum possible rate, removing bottlenecks, etc), scouting the enemy army composition to prepare the ideal counter army composition
- Tactics: Grand army decisions - flanks, baits, sneak attacks, hiding composition, multiprong attacks, positioning of siege units, timing attacks, knowing when to retreat (hit & run), scouting to gain advance notice of your opponent's tactics
- Micro: optimizing unit lifespan and effectiveness within an isolated skirmish for the given goal (usually to 'win' the engagement) -- pulling back weakened units to avoid aggro while it still deals damage, healing, surface area for melee units, trapping enemy units with terrain or skills, optimizing spellcaster energy usage, prioritizing targets based on multiple parameters (range, damage, cost, count, follow-ups), etc.
AI can "run" macro well, but they are poor at the macro decision-making part, which includes priority model as you mention (the responsive posture choice above and others). Up to low grandmaster tier, being significantly better at macro than your opponent while close in tactics and micro is usually enough to win consistently. It is the most impactful part of an RTS (and is where most of the 'S' lies).
That said, in high-level play, "better macro" usually refers to the other things, not just mechanically hitting stride with production, as most players in the top .01% are on the same level with those mechanics.
But if someone was microing a battle and as a result didn't look at the minimap and see a drop arriving at their base, I can totally see concluding that this player is bad at macro as a result -- macro is referring to there being a macro cycle of tasks you have to perform all the time whether you want to or not, and non-production tasks like checking the minimap and sending in a scout seem like good examples of those tasks too.
Execution of the answers to those thoughts is in the form of tactics and micro, the other two aspects of macro. If you're following current SC2 pro meta, a "strong macro player" however has more right answers to most of those questions (Stats, Innovation) and that's their strength, versus a "strong tactical player" (TY, sOs) or a "strong micro-based player" (ByuN, herO).