OpenAI Five Benchmark: Results
blog.openai.com
blog.openai.com
The team of semi-pros/ex-pros/pros that played against the AI commented that the AI was using the highly unusual 5 invincible couriers to enable a style of play that isn't possible in normal dota. The AI's solution to dota was unrelenting aggression once a small early game lead was established after ~10 minutes. This early aggression was possible because the AI was able to ferry items (healing consumables) with such frequency that once the human team made one mistake in a team fight it wasn't possible to recover.
Normally after winning an objective you are forced to reset as you have expended a lot of resources to order achieve your objective, and you are actually most vulnerable to a counter play right after you win something big. With a constant stream of healing this risk was significantly reduced.
Also, the drafting is a clear limitation that needs to get lifted. The AI was essentially pursuing a "death-ball" strategy of grouping as 4 or 5 and pushing right down a lane, which can be countered by picking mobile and fast heroes that can put pressure on different parts of the map and slow down the "death-ball" by forcing reactions. However, none of those heroes were draft-able, and so the humans were forced to play the AI's strategy. The AI's strategy favors team fight coordination (laying stuns correctly, and correctly calculating whether damage through nukes is sufficient to kill any particular target) and reaction times, at which the AI was clearly superior to the human team.
I'm just slightly bugged by the fact that the developers didn't take action execution into delay consideration. The response time is 200ms but humans also need some more time to drag the mouse and click to perform the action. Their insane reactions actually make me less impressed.
Regarding the insane reaction, I wonder if there is a natural way to handicap the AI to more human-like reaction times. Reaction time is not a good measure on its own because human reaction time can vary a lot depending on the level of surprise.
That said, my impression from watching the game was that the power of the AI had less to do with perfect reaction time and more to do with their "hive mind" coordination. If an enemy ever gets in the wrong position, they are immediately punished by a concerted attack. Humans have a harder time doing this because they need to communicate their intentions first. Sometimes each player will be focusing their attention on a different target.
> That said, my impression from watching the game was that the power of the AI had less to do with perfect reaction time and more to do with their "hive mind" coordination.
Yes, which is why they restricted character selection to favor it. There's no reason to call it a hive mind as it doesn't possess that. There's a global steering function presiding over 5 bots that need to act swiftly based on global dmg/etc. This is why they restricted the game to snowball cheese. A human can't beat this just as a human can't beat a TI-89. This is why, if you look closely, they absolutely destroy the bots when the team is separated and the human players aren't playing like greedy noobs diving towers.
> If an enemy ever gets in the wrong position, they are immediately punished by a concerted attack.
> bot : (10-9) =1 (I win) Strong AI right around the corner
There was a moment in game 1 that was the exception proving the rule for me. The bot playing lion successfully disabled the human initiator on earthshaker at the end of the game. It looked like a superhuman reaction, but it was also a bit different from all the other fights of the game where it was usually the fundamental position being too far in the AIs favour - they had a gold advantage and had been developing a lead through the entire game by consistently trading deaths 1-0, 2-1 or 3-2 in engagements.
The potentially superhuman reaction took the game from "looks like bots are winning" to "humans resign now", but the vast bulk of the advantage was that the bots simply had a better understanding of which team enjoyed a superior position. I would not be surprised if higher bot reaction times (+100-300ms range) weren't all that impactful on the results.
It'll be really interesting when the courier distortion is removed and the AI has to play more defensively. Also, I suppose the actual, harder to articulate, complaint in the "reaction time" complaint is that the bot teammates have the capacity to chain abilities more accurately and have played so many hours together there is an advantage there that isn't 'fair'. It'll be a fun milestone when they can drop a single bot in a pub game where their teammates aren't all that coordinated.
But yes, very exciting next stages when there are item builds, more heroes, and standard couriering.
I'll add:
- The coordination between AI bots are clearly beyond human level. Or at least as demonstrated from similar performance from the humans on similar style of heros.
(It might not appear too different from the show match, but based on my 2-3k hours watching pro games, the coordinations are noticebaly better than the best team in history, aka Wings gaming 2016 TI champion).
I am not sure how such coordinations are modeled in dnn, which itself seems the most valuable from this research.
- In general I think with this show match, it pretty much sealed the doom of human players in dota2.
As it shows that the general approach is scalable and capable to handle the problem itself. As from laning to team fight, and item building, the AI did not show weakness at all.
I was worrying about AIs general inefficiency in deriving the winning strategy, laning stage, and team fight coordinations, which turns out to be obviously superior to human players.
Drafting probably will be even more favorable to AIs. The challenge would be can they train faster by observing the change log, I.e. finding winning strategy without training from scratch each time after a patch release.
I seem no reason AIs lose to vp/liquid/lgd (the top 3 going into ti8). The idea that split pushing hero can deal with the team fights seems underestimate the AIs discipline, which is clearly superior to the best humans.
- Last is how much computing resources are used in the training and playing. Hopefully value can team with open ai to release a benchmark bot team for calibration and a different ladder systrm of playing different AI strength level
I was disappointed to hear one of the devs say that item selection was hard-coded following a popular guide (don't think they communicated this before).
This is OK, because what matters the most is that it won. It was previously impossible to build hard-coded AIs that would even beat decent players, and now the AI has beaten some pro players (albeit not the best of the best); but it'll still be nice to see superior item buying strategies being learned.
Also, computer engines didn't seal the doom of human players in chess and in go, so I don't get why it would do so in dota.
Which I'm still not completely sold on. It's likely, but the remaining restrictions aren't trivial by any means. There's an outside chance that removing one or more of them is going to brickwall their progress.
I genuinely believe the bot would win a game of turbo against any team in the world. But remove _all_ of the restrictions and it's not clear that it doesn't just lose at the moment
So you're saying that even before they set up a rule about microing illusions to protect humans from a feature that they have not yet implemented nor, I assume, have trained the model on?
As far as I know it is five (Hence the name) individual AI instances controlling each character and with basically no AI to AI communication.
It is not one overriding AI controlling all five.
I have no idea if the AI instance controlling each character is identical though, if so then your statement still holds true I guess (Assuming each AI has the exact same information to work with which might be the case). It would be interesting to see if AIs specialised.
As a counter bot strategy, I'd work on how to break and trick it using multiple-stepped logic that an optimization function would be unable to see beyond. I'd also use varying tactics of chaotic/sporadic configurations. The bot isn't 'playing fair' nor should a human w/ intelligence. The advantage being that a human can think along a multitude of strategies and adapt. The bot is only optimizing some steps ahead.
Their 1v1 bot was defeated in this manner and it just goes to show what true intellect and superiority is. I've played random pub games w/ little to no communication and have had all other 4 players converge on different strategies based on a perception of what's going on. If someone decided to cheese/snowball, you simply wait it out and let them push themselves into a nightmare. I saw little to none of this in the games I watched which leads me to question the intelligence of said 'pros'.
The actual source of the "communication" is not the team spirit parameter, but the basic fact that the bots have been trained together and they receive the same inputs when making decisions. Unlike humans, who have a limited focus to their attention, the bots can look at the whole map at once. They don't need to communicate because the already "know" what their allies will do when given the same input.
That said, the AI still won despite these limitations, which shows that it is very strong in other areas of the game.
Not everything can be calculated especially when tricks are intentionally done to throw a bot off.
> I am not sure how such coordinations are modeled in dnn, which itself seems the most valuable from this research. There's a tuned group/individual driver function centered on various calculations. This is not actually a valuable part of the research as its dynamic and game dependent and can't cover all of the possibilities thus why someone broke their 1v1 bot (corner case)
> In general I think with this show match, it pretty much sealed the doom of human players in dota2.
If you are indeed a player and have viewed that many hours of dota2, I question the nature of such a comment. A great player wouldn't look to see how to 'beat' their bot, as it is a bot w/ no intelligence, the strategy would instead be to try to break it and shove it into corner cases. It's not playing the full range of characters that were intended to disrupt cheesy snowballing so I wonder why you're making such an optimistic statement .. being that you claim you have watch so many dota matches. Do you play much yourself? Maybe that would change your opinion.
> As it shows that the general approach is scalable and capable to handle the problem itself. As from laning to team fight, and item building, the AI did not show weakness at all.
I'm starting to see a pattern with your commentary. The gameplay look like your typical "south" players. Hardly anything impressive : Aggressive boneheaded tower diving and aggressive and cheesy snowballing.. If you can last past 30min, you outwit and outplay such people in the mid/late game.
> superior to humans > superior to humans > superior to humans
More than 50% of the Dota 2 dynamics aren't even present and are restricted at the moment. Are you getting paid for this post?
I cannot see any differences between "beating opponents" vs. "break it and shove it into corner cases", that's always how dota games are played out.
Unless we formed different views on how Dota is played through our thousands of time watching games (I sensed from your statement that you had similar gaming experiences :) )
This is what Strong AI is centered on.
AlphaGo Lee (the version which won 4-1 against Lee Sedol) did seem to get thrown off track by Lee’s surprising move and lost that game.
However, AlphaGo Zero, which is based on some of the same principles/sets of algorithms, were much stronger than AlphaGo Lee (More than 3 stones according to DeepMind. Three stones is about a difference between top pros and top amateurs/beginning pros.) and seemed like it would be insusceptible to any surprises thrown its way from human experts.
The difference was that AlphaGo Lee learned from play records of human Go experts while AlphaGo Zero did not and only learned via self-playing. Dota 2 is clearly more complex than Go but if the same principles apply then an AI trained from pure self-plays would be adaptive to most surprises in the domain, if the system had explored those edge cases before (which depends in turn on how the self-plays were conducted during training).
(As a side note: OpenAI Five probably chose the “simple-minded” snowballing-cheesing strategy because it determined from extensive experience that the strategy is most likely to yield a win given its capabilities (which are advantageous to humans in some respect like instantaneous global information observation, great coordination, consistency, etc). This is very different from the reason some human players choose the strategy. Perhaps precisely because Five bots don’t get sloppy that the strategy is so effective for them.)
Most pro games are played with a strategy that is settled once draft is finalized. If the strategy turns out not working, Humans did not show noticeably different adaptivity.
Occasionally, a versatile team can transition from a late-game oriented line up to play a split push game. But usually such transition is based on a suiting draft, which requires the team members to be versatile in playing their heroes in slightly different styles; and a well-oiled team coordination to transition from one to another style.
> the “simple-minded” snowballing-cheesing strategy
In the show matches, there is no cheesing. It's plain team fight + push; the AIs executed the plan with ruthless precisions.
TBH, a typical pub game is best described as strategy-less game play. And pro games probably have 3 styles of play:
- Team fight
- Stick-together push
- Split push
The most close team that shows vastly better versatility is Wings gaming, which pretty much run any lineups they feel fitting.
Sadly the team disbanded after TI6, otherwise, their match against with OpenAI would be the most interesting thing I can imagine.
They are. I'd actually argue that this occurs much more in a pub game than with pros. I'm largely against the concept of a pro for this reason as it amounts moreso to having settled on lockin strategies moreso than intelligent/active/dynamic exchanges. I play a lot of pub games for this reason... To enjoy the heightened dynamics. Tons of rotations and adjustments. Tons of punishments for a great player hot dogging to break their psyche. Lots of very intense examples of dynamic human intelligence.
> Most pro games are played with a strategy that is settled once draft is finalized. If the strategy turns out not working, Humans did not show noticeably different adaptivity.
You're speaking moreso of 'pro games'. I encounter a great deal of dynamics outside of this grouping... It's where a lot of intelligence comes into play. I think a lot of people are completely uninformed about the game who watch others play a lot w/o actually playing themselves. Pro games are literal theatre for the masses like in a large number of professional leagues. The real stuff happens outside of the spotlight.
> Occasionally, a versatile team can transition from a late-game oriented line up to play a split push game.
This happens in just about every game I play... Tons of rotations and readjustments when things aren't working out [happens sometimes w/ no communication]. Tons of split pushes.. Strategic ganks. If people have good emotional stability, there will be a pronounced reflection/change after a massive team death incident.... The point of these games are highly intellectual battles. Its why it's a disservice to restrict any features of the game. It's how they maintain balance to avoid the game devolving into idiotic bot like cheesing. Games live and die based on how much cheese is present.
> But usually such transition is based on a suiting draft, which requires the team members to be versatile in playing their heroes in slightly different styles;
I'd expect pros to have these skill-sets yet I don't see it much because in such showcases its more about optimization and lockin strategies than dynamics.
> and a well-oiled team coordination to transition from one to another style.
Happens in regular pub and ranked matchups all the time many times w/ little to no communication. As long as someone is not an emotional child, it can sometimes be stressed and instantiated over prolonged swearing and yelling at various players. This is what's maybe missing from the Pro-league... Someone getting in your ass openly for doing something stupid like continuing to battle 3 well organized bots 3v1.
> In the show matches, there is no cheesing. It's plain team fight + push; the AIs executed the plan with ruthless precisions.
One of the games opened with 4 bots diving bottom tower to get a kill and persistently pushing bot. The human 'pro' sat there hashing it out even though he could have ran to safety and avoided another death and no one from top or mid tp'd to bot on the human team. This hamfisted cheesy snowballing occurred in every match on the bot's behalf because OpenAI restricted the gameplay to favor it. Even so, in a pub someone would have been swearing to the top of their lungs on a mic telling the $@(#@(%* at top/mid to immediately TP and punish such a brazen exchange especially with creeps all over them. Absolutely nothing was precise about the gameplay from the humans or bots. It was the kind of slop I see on servers from the southern portions of the world and punished heavily by any seasoned players. I guess this is where 'pros' are a meme and I've served a good number of them up with gameplay outside of their carefully scripted comfort zones.
> TBH, a typical pub game is best described as strategy-less game play. And pro games probably have 3 styles of play:
Typical pub is chaos which is why I've seen a number of pros get their behinds handed to them in it. They're sort of like bots in that they think they have the game completely figured out and have a golden strategy no one can defeat. It's a flaw not a good trait. In ranked, you're going to see some amazing gameplay even w/ random non-party individuals. Anyone who plays knows about the games where its like a symphony playing. Limited talking, tons of rotations/ganks/readjustments/team fights/split pushes/team pushes/ratting/baiting/etc.
"Pros" are not Pros in my book. They're a group of players who center on a optimal echelon of gameplay that everyone at that level tends to agree upon. Throw some dynamics in and they fall apart.
What I saw across all of the OpenAI bot games is nothing to go home writing about. If they were true to things, they'd show how these bots play in all-pick no restrictions. They claim to be after Strong AI not Weak based game bots. It's not about winning/losing... It's how you play.
This is enough of my personal commentary on this issue. People are unable to see past these approaches and what they truly are and that's fine with me at this point.
Catch you on the flip side.
What would really happen when pub-styled play is actually used in pro scenes for million-dollar prizes? If it is actually superior, why none of the pro teams caught up and tried using it to win?
Thus the nature of a canned showcase demo. We do know they have a slew of restrictions. As an avid player, I know exactly why : because such combos require much deeper and true intellect to play efficiently. Even as such, given that I know i'd be up against an optimization algorithm, my strategy would be to create as much chaos and uncertainty as possible. Information theory is clear as to the impact this would have : It would be unstructured noise that would be hard to optimize and likely not seen before or significantly reflected in the AI's weighting system. This is the basis of adversarial attacks. I'm sure with a decent amount of games I'd be able to figure out a suitable one for 5 linked bots.
The perspective as to what's going on with this demo is much different if you actually play the game. I've actually seen a number of games like this bot exhibited. It's a strategy low skilled players engage in with the hope of overwhelming opponents with brute force. The character restrictions favor it. So its not by accident that this all converged into a demo that favors an unintelligent brute force optimization bot.
It favors something that can do range/hit point calculations quickly/accurately. Snowballing is required because there is no broader intelligence among the bots. When the bots snowball, it's essentially just one big optimization function. When they're stretched apart, the calculations are much harder.
Knowing what I know about the game and the fact that I'm up against a Weak AI bot with an optimized model, I'd know exactly how to screw it up with an adversarial attack. I'd train a team of people on that and show everyone exactly what human intelligence of capable of and why its superior. This happens in your average dota 2 match constantly.. Low skill players attempt brute force strategies just like these bots and you essentially wait them out and pick them apart. This isn't a new and amazing style of gameplay or something. There's already names for it.
When I used the term 'sloppy' I meant against the spirit and nature of the game and w/o consideration of the 'way in which one wins'... Ambushing towers at open 4v1 or 2 is some very hamfisted foolishness. Even in regular pub games with upper avg. players, there'd be a sharp punishment for such bro-tier gameplay. It usually results in an equally massive 'gank'. The way the human players responded in these pressure scenarios really has me questioning the whole event as I see avg. random players make far better decisions every day in dota.
That's just my unfavorable two cents. I'm not impressed because I understand how their bots are doing what they're doing, where the advantages lie, and I'm aware of what restrictions they placed on the game in favor of their bot.
Elon claims he's worried about a dark future with AI, it's actually solutions like this that are most scary because there is zero intelligence and a [by any means possible so long as you achieve the object] steering function. If you want to unleash chaos and destruction on the world and see a darker side to human intelligence you've never seen before, start releasing such 'weak AI' to manipulate people from the shadows. This is not strong AI or a path to it. It's more of the same Weak AI provided with exclusive and insane amounts of computer power/data and an objective to optimize for by any means necessary. In cases where it dominates, it's almost certainly a reliance on finding loopholes/flaws in a particular game not actual intelligence. You should see the danger in this right away.
Funny because OpenAI originally opened with the spoopy terminator like dangers of AI being so destructive we needed a group like them to save us... To now openly sharing such unintelligent and dangerous weak AI optimization platforms in the mainstream fear. Sort of like the 'Do no Evil' Mantra that was just slogan.
I think this is a great engineer accomplishment that no doubt taught them a lot. I don't see any broader 'safety' ideology underlying this... Just another great team of people trying to achieve AI like everybody else utilizing popularized approaches. It's better to just come out and say that. We can drop the 'Save the world from AI'/'Safety' superman talk and get to the brass tax of what they are doing and how, if at all, its different from what anyone else is doing in the space.
As a dota player who has been in the 99.5th percentile mmr at several occasions (right now at 98th) I disagree with this and a lot of the stuff you're trying to say. Dota is a strategy game, and the meta dictates what strategies are strong at a given point in time. The death ball strategy that the bots played is a result of that being the best strategy in the bot meta. So in contrast to what you said, it's not low skilled players that play these strategies, but rather high skilled players that play whatever strategy is popular in the meta (regardless of how 'intelligent' it is), in order to increase the chances of winning.
Game 3, the OpenAI was forced to reveal its carry pick (HARD CARRY SLARK) as first pick. What a farce.
The one thing that really surprised me yesterday was when OpenAI Five seemed to pause the game. The commentators speculated that “it was learning” because the humans had paused in game 1 due to lag.
I assume that’s not right, as OpenAI five is not training itself as it’s playing (wouldn’t make much sense to add one more game sample to the billions it has already trained on).
I thought it was interesting that the commentators had this misconception, and was wondering what lead to OpenAI Five hitting pause.
Incidentally, we'd just changed the code a few days earlier from "automatically surrender when a human disconnects" to "do nothing if a human disconnects; automatically pause if all humans disconnect". Had done a lot of advance planning for what might break!
Folks in the DotA 2 community are going crazy over this possibility, would be incredible if this happens.
[0]: "These results give us confidence in moving to the next phase of this project: playing a team of professionals at The International later this month." from https://blog.openai.com/openai-five-benchmark-results/ at the bottom
It was a really wonderfully done event overall and there's several interviews throughout with different folks from the OpenAI team.
In chess, both players always have perfect information about the game-state, and this is far from the case in Dota. OpenAI does account for fog of war, so it's not COMPLETELY omniscient, but it is still more omniscient than human players ever have the ability to be, without having to fiddle with the camera etc.
You wouldn't want AlphaGo to have to input it's commands using robotic hands right? It's the same thing in that, sure it might be interesting, but that isn't what we care about. Image processing and robotics controls are largely solved. Showcasing that a model can gain the ability to plan and think is the novel stuff here, and is the path to where "artificial intelligence" if any appears. That's the ultimate goal in playing any of these games.
Image processing and robotic control are very far from being solved problems. I guess you are saying that in the case of alpha go it would not be a super difficult step to have a camera and robotic hand physically move pieces around, and that's probably true. But I think in the DOTA case are new image processing challenges that interact with the AI in interesting ways.
I'm mostly talking about the need to move the game's camera around to gain more information. If you don't see your ally on your screen and need to see how they are handling a gank or something (full disclosure I don't play DOTA at all this could be a silly scenario). Then the AI would have to recognize this and move the camera to the allies location in order to gain that information. So really the novelty here would be in the network to somehow realize what information it needs and then further to learn how to gather that information. I honestly think that sounds like an extremely difficult next step.
Humans dont concentrate on the whole screen, attention is directed...
I would love to see camera/mechanical interface like mentioned by others. Similarly, like you said humans don't focus on the whole screen. I would love to see how well the AI could perform if it was given something like blinders where only a small portion of the screen is in focus at any one time much like how human eyes work.
Otherwise, you still have to add time to react plus mouse travel time.
That said, everything they've done so far is absolutely incredible (especially now that the AI can draft!!)
Basically, they let an AI loose on a simplified version of Quake's Capture the Flag. The AI processes game video output only and has learned several key strategies. The latest update has the AI with a winrate of 71% against top humans. Unlike the DOTA match the AI has no restricted reaction time.
The AI seems to be jittering the camera left and right to reconstruct a 3D image reliable from the screen which is aquite interesting way to compensate for lack of 3D vision (and the compensation our brain is capable of naturally to get a 3D intuition from a 2D image)
If this was actually the goal, they would add further mechanical restrictions beyond the 200ms delay to simulate the way humans players play. That way, human and AI would be on a roughly even mechanical playing field, leaving the differentiating factor only strategy/tactics.
As it stands, it looks like their victory is as much based on raw mechanical superiority as it is strategic/tactical superiority. Computers being able to be pixel-perfect accurate at all times, issue commands at ludicrous speeds, etc. is kind of an uninteresting advantage in the context of building strategic AI.
For many things in Dota you also need to move the mouse cursor to a specific point on the screen which obviously takes longer than just pressing a button.
So this current ai is uninteresting because bots can always instantaneously begin to react on any feedback, whereas humans have to pan and drag the camera around to look at different feedback in the first place, let alone react. Mechanically, humans also have to move the mouse all over the place and think of key combinations, in addition to reacting. Not just clicking a static box on cue.
It -would- be interesting if bots were limited just like humans to the camera view, -not- an API that continuously feeds them information. The bot would then have to learn how to prioritize working the camera, and it would be limited to only what the camera sees, etc.
Computers already have perfect memory and recall, so when the image recognition tech becomes good enough to only rely on the visual input, are you then going to say the bot must now limit its recall to "human" levels?
The problem, as they stated in the QnA is that the image processing would take much more hardware and computing power, this increasing both cost and training time, as you would not be able to run games as quickly anymore.
Given the massive network architecture linked in the post (https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchm...), I am rather curious what hardware was used to make predictions for the Benchmark match. Especially due to the 2048 unit LSTM.
The model outputs are also interesting; they're all discrete actions (even movements), no continuous outputs (aside from win probability).
Although making discrete choices 2-3 times a second is indistinguishable from continuous movement anyways.
Great results that deserve congratulations- but they show nothing of the sort. The distance between a game and the real world is about twice as big as the distance between a game and a simulation of the real world (which a game isn't even). Winning at dota is nothing like negotiating the real world.
I'm not sure what will follow.
I occasionally play strategy games and I am always frustrated by a very stupid AI that gets challenging only by beefing up its stats. I would love to play against a smart AI.
I don't do multiplayer because: I like to play offline and/or whenever I want, and I play so rarely that most other human players wouldn't enjoy playing with me.
Open.AI team: please listen!
Why was Dota chosen as the game for an AI to get good at?
Is Dota more difficult than Go? Why or why not?
Dota 2 was chosen because it's a nice combination of pre-determined rules (to an extent) and an extremely complex problem set of possible moves and actions. Dota's development team also was really supportive to this effort and they have an API that suits the Open AI team's purposes of interacting with the game really well. Great fit all around.
Dota 2, especially when played 5v5, is orders of magnitude more complex than in terms of possible decisions and moves than Go.
1) In Go, you are allowed to see the entire board and pieces at all times -- it is a complete information game. On the other hand, games like DotA have partial information because you are not able to see where your opponents are and what they are doing at all times.
2) Go, Chess and many of the Atari games are single player games. OpenAI wanted to see if machine learning can be applied in a multiplayer setting where the problem needs to be solved at a global / team level.
3) DotA has so many mechanics and strategies where you have a lot of choices to be made. One of the challenges is whether one can look at the overall outcome of the game and reason about what particular choices went into the winning or losing of the game. This (long event horizon) makes it extremely difficult to learn such models.
OpenAI interfaced with a complicated game like DotA through an API provided by Valve which made it lot easier. Instead of seeing the game screen, they got snapshots of data of around 35KB per observation (co-ordinates of heroes, creeps etc). In the absence of this API, they would have had to use substantially more computational resources to render the in game graphics and this would also make the training process extremely slow.
This benchmark was an experiment that demonstrated that tackling such a class of problems is indeed possible (given a lot of computational resources and an environment to train in). During the interviews in between and after the games, they mentioned that the algorithms that they have used can have many applications in all fields a̶l̶t̶h̶o̶u̶g̶h̶ ̶s̶p̶e̶c̶i̶f̶i̶c̶ ̶e̶x̶a̶m̶p̶l̶e̶s̶ ̶w̶e̶r̶e̶ ̶n̶o̶t̶ ̶p̶r̶o̶v̶i̶d̶e̶d̶ @crsv's comment describes the example they talked about.
Perhaps an ELI15, but hope this helps.
The first being that Dota is a team game which requires teamwork to win. This presents a new challenge of having actors work towards both personal and team goals in a balance (much like real players) to be able to win, which is very difficult to train.
The next is that Dota and Go are two very different kinds of games. Go is an "information complete" game, where all players have access to the entire game state at any given time. Dota, on the other hand, is an "information incomplete" game, as teams are vision restricted: there's no guarantee on the state of anything out of vision, meaning that the AI has to develop what most players call "game sense" in order to be effective.
On a tangental note, it's also an interesting problem from a state space perspective. Go is technically "solvable" to a point where you could (with a currently unobtainable amount of computing power) find an optimal move, but Dota is almost unfathomably more complex: 10 heroes picked from 115, each with the ability to hold any combination of 9 items from about 150, with abitrary health and mana values, at abitrary positions on a large map, not even mentioning the non player units (creeps and neutrals). If Go's state space is our solar system, already a difficult scale to comprehend, Dota's is the whole galaxy.
[Typed on mobile, apologies for any typos.]
I don't think this is actually a significant problem for an AI, because each AI 'player' will be the same copy of code, thinking the same way. They don't need to explicitly communicate if they have the same thoughts and expectations.
For AI, yes. It's real-time and there's a fog of war (no perfect information). The former means that the possible action/choice space is vastly larger at any given moment in time, and you have to draw a line between small/meaningless differences in actions (e.g. a one-pixel difference in movement target generally won't be significant) and meaningful differences. The latter means that the AI needs some kind of mental predictive model where it guesses at what the opponent has likely done in the time that their actions have not been visible.
One interesting thing about Dota is that it is a slower-paced game which isn't heavily reliant on twitch reflexes. To win the bot needed to show good teamwork.
Dota2 also has a developer-friendly "bot API" and replay system. I'm certain these affected their choice of game as well :)
Are the AIs actions rate-limited, delayed, and perhaps time randomized a bit so that an AI doesn’t get an advantage because it can to actions faster or better synchronized to exact times, or more responsive (say to avoid any action under a certain reaction time, all actions are scheduled 50+rand(5) after the AI queues them)?
I think first it was planned for 28th now i read it one day later :/
That said, you still have a chance to catch the match against a professional team that is going to happen sometime between the 20th and 25th.
https://elie.net/blog/hearthstone/i-am-a-legend-hacking-hear...
People need to start doing more critical analysis of their own and stop relying on commercialized and biased information and commentary when settling on an viewpoint. Watching others play only gets you so far in understanding. When you play and you see what I'm saying for yourself, you can skim through the provided clips and understand exactly what's going on.
The fact that this continues to get hyped up vs someone stating what's going on is plain sad.
As for Dota, indeed the broader game was designed and is constantly updated to be an intellectual challenge. Cheese strategies exists. When discovered, game rebalances are conducted to ensure players don't settle in on brute force exploits.
In just 5-10min of watching some of the matches, I already know a handful of characters that would 'break the bot' on this 5 bot ensemble. This is powered by human intellect (the truly significant part). The bots trained across time horizons greater than a human lifetime in this unorthodox and slanted arrangement. Human beings seasoned on a more well rounded and balance game are then thrown to the bots and showcased. I know who would be sold on this and its not your average Dota player.
Again, the fact that it is possible to 'break the bot' examples that the whole thing is a charade. I have no doubt they might have gained some experienced/understanding with engineering this solution but the actual result is lipstick on a 'backpropaganda' pig.
More fundamental deep research and less theatrics is in order. I'd also advise becoming seasoned in the thinking behind a particular game if you're trying to test and develop an artificial form of intelligence to navigate it and beyond.
You're 100% right and being down voted for it as I no doubt will. I am not impressed because I've played a ton of games and observed a ton of dynamics. You can tell in the first 10min what level and type of game play is occurring and it isn't of the intellectual variant. It's of the bot variant. Broader Dota was designed and constantly updated to combat such gameplay. When people settle on cheese strategies or exploits, they intentionally change the game balance to prevent it.
I'm more interested in how this is achieved than what it has been tuned to try to trick people into believing. Looking under the hood, I see the same thing when I look across all Weak AI solutions : Lots of hand coded steering functions with dynamic weights populated by incredible amounts of brute-force random searching (past a human lifetime of average gameplay).
While this may secure headlines and funding, this is miles away from the real direction you have to go towards Strong AI. It doesn't seem many well funded/popularized groups understand this thus will ultimately end up fooling themselves. Point out this truth and you get down votes. Essentially the broader community plugging their ears which is why a large number of these efforts are due for a significant failure.
As for your point, we’ll have to see what things look like as OpenAI continues to rapidly remove these restrictions! Or you could provide us some literature on the broad strokes you’re making.
My points stand on their own. I take downvotes w/ no sound rebuttals as a signal that I'm saying something that right and an inconvenient truth.
> Or you could provide us some literature on the broad strokes you’re making.
I began my work by not pretending that weak AI is something that other than what it is and prioritized the more challenging aspects of AI