Hacker News Reader

Comment by reroute1 | Hacker News Reader

> "By defeating theDota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcementlearning can achieve superhuman performance on a difficult task."

Isn't this a bit of a leap though, it came with massive caveats to the game:

1. Drafted from a 17 hero pool. 17 out of 115??

2. No summons or illusions. Again drastically reduce possibilities in game.

3. This AI is trained against real players for years, so it has enormous experience against this type of opponent. The opposite applies to humans who never compete against bots, so have no experience against this type of opponent. If I recall correctly the more people played against the bots the better humans performed in successive games. Winning two games with all these restrictions and caveats is still impressive, but it feels like they overstate things. Not to mention the flawless mechanics and communication between the bots...

View on HN

People tend to focus on what has been left out, but think about what they actually did learn about:

Drafting from that pool, item and skill builds, last-hitting, creep aggro, laning in general, jungling, item and spell usage, ganking, team-fight positioning, pushing objectives, warding, map control, farm priority, when to retreat vs engage. All of these require an understanding of micro vs macro goals and how they relate.

Surely this qualifies as "a difficult task."

Dota's built-in bot can already do most of that

Dota bots are useless.Can be easily exploited and defeated.

And that's being said by someone who is on the worse send of bell curve :)

> This AI is trained against real players for years,

AIUI the bulk of the training was self-play.

> Not to mention the flawless mechanics and communication between the bots...

The bots had no communication channel.

> The opposite applies to humans who never compete against bots

That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches.

> AIUI the bulk of the training was self-play.

Maybe but it also has played against humans in many instances across many years time span.

> The bots had no communication channel.

The execution between them was flawless though, acting as one mind whereas a team of human has to communicate ideas. It's a clear advantage, but it just feels hacky in a way since it's not really comparing the same thing. 5 humans are a diverse group of people. Maybe it's not fair to knock the bots for this behavior though.

> That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches.

I'm talking about playing against OG, who didn't spend days playing against bots. Beating regular players is great, but not the accomplishment they are portraying. The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results.

> Maybe but it also has played against humans in many instances across many years time span.

Those are evaluation matches, they don't feed into the training data.

> The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results.

Well, the pros could have joined the open games too, I don't know if any did.

Yea I think you are right about the training data. So that would be more impressive. I still would like to see a team dedicate some time to it, but that's really not in the interest of a pro team. Also the things that affect real tournaments like drafting would be really interesting to see some day also as that is a huge part and considered a more cerebral aspect of the game.

> The execution between them was flawless though, acting as one mind whereas a team of human has to communicate ideas.

A team of humans that has played together extensively will often intuitively know what the other human will do in a given situation without explicit communication. The fact that humans can also coordinate in unusual circumstances where intuition is insufficient is an advantage that the current version of the AI lacks.

I don't think it's overstating it too much. Even with the restrictions, the bots are still performing well above the capabilities of the vast majority of human players. The restriction on illusions and summons was also for the benefit of the players; the OpenAI team didn't want the bots to win through flawless micro skills. For point 3, even though the players didn't get to train against bots, they have the advantage of being able to learn and react accordingly. Since the bots can only learn when their models are being trained, they're trivial to beat if you use a novel technique that they haven't seen before.

If you want the bot to avoid winning by micro, you add delays, cooldowns on interactions, imperfection on clicking, misclicks, etc. on the level of a human pro.

In any case, simplifying the game is usually done to make training far cheaper.

But the human competitors also require large amounts of training in order to competently play Dota 2, and their training is not simplified that training in a similar way. I realize that "fairness" is not really the point of having humans play against bots, but doesn't it damage the usefulness of the comparison from having them perform a human activity?

The paper mentions not using illusions and summons because of the added technical complexity required to make them work:

"We removed these to avoid the added technical complexity of enabling the agent to control multiple units."

So maybe it could be to their advantage, but it's also not something that could be easily technically accomplished at this time?

The work I've seen recently from them on multi-agent play seems to indicate this is a problem they're very successfully working on. https://youtu.be/kopoLzvh5jY

just that by not leaving players walking randomly is already an astonishing accomplishment to my simple eyes.

Plus, the match happened right after OG had roster changes, probably the weakest point of the team's history, well below the later achieved TI champion status, which is long after this match was held.

Well, anyway, I am OK with some degree of PR additions...

Reply on news.ycombinator.com