Hacker News Reader

Comment by dvt | Hacker News Reader

Pretty much agree with everything in here. As I said in my earlier posting (and this blog post reiterates), a 1v1 Shadowfiend mid is highly technical and does not require a huge search space (like in Go or Chess) or any judgment; all it takes is a few tactics (e.g. creep blocking) and good aim for the razes.

Also, the bot was already beaten 50+ times[1]. There are at least 3 strategies that work. It just goes to show how primitive AI is, as it took the AI team thousands of generations to get it to this stage, but a few determined gamers outsmarted it (using a few cheap meta-strategies) it in less than 6 hours after release.

[1] https://www.reddit.com/r/DotA2/comments/6t8qvs/openai_bots_w...

View on HN

If you look at this problem realistically it should be offensively easy for AI. The entire world state is available to you with perfect accuracy at a timescale that is trivial for computers but well beyond human reach. Also, there are clear goals and absolute limits to the functionality of the world.

But the human players quickly adapted and developed new strategies and the bots just weren't able to adapt as quickly. You can see the players recognizing the idea of a training set and trying things the bots probably hadn't seen before to see if they could confuse the bots.

As much as there is a lot if impressive work being done in machine learning I can't help but to be generally a bit skeptical of the whole "AI revolution" and people being replaced en mass. I'm far from an expert but from everything I've seen from machine learning seems like this awesome tool to augment human ability rather than replace it.

I'm not super interested in having a computer doctor but I'd be 100% on board with my doctor having smarter computer.

I think the greatest challenge to date in machine learning is identifying how to use it to create value.

If AI can augment the ability of 1 human then that 1 human + 1 AI can still replace people en masse. That's what automation does. The factories usually are not devoid of people, they just have fewer people and higher productivity than they used to.

This whole OpenAI gets hyped a lot on HN - it's interesting who is behind OpenAI. They are misusing the "Open" name as there is no substantial open in OpenAI - it's also no open source project - it's not related to OpenGL, OpenAL, etc at all!

I don't get the hype around AI bots for StarCraft 2 and DotA 2. There is nothing special now, we had such bots developed for many years.

https://github.com/openai

While not everything has been opened, they do have a bunch of things they've opened up, particularly the simulations for training AI in.

Have you seen any bots perform competitively with human pros in Dota, even in the easier 1v1 case?

We have had hard-coded bots that perform decently well. We haven't had ML-trained bots which perform at this level which is what is exciting about this. It's not about having a computer play DOTA at a high level, it's about having AI learn something in a real-time environment, as suppose to turn based like go or chess.

Yeah, but the method of learning is interesting - unsupervised training in a VR world. As CPUs become faster, than these type of problems will fall much faster.

I am not sure it is completely unsupervised, iirc they said it was "coached". I don't think a bot can learn by itself to fake razes/perfect creep blocking by itself.

They talked about this twice, the exact text of both follows

> We didn't program it to understand the rules of Dota, we just let it play lifetimes of 1v1 against itself and coach it on what we thought was good or bad.

> We've coached it to learn just from playing against itself. We didn't hard code in any strategy. We didn't have it learn from human experts. Just from the very beginning it just playing against a copy of itself. It starts from complete randomness and then it makes very small improvements and eventually reaches the pro level. [... goes on to talk about how it's first "improvement" was to just never leave the base because it kept dying]

I think you're reading too much into the word coached. Particularly the second quote implies that it wasn't supervised at all.

My personal explanation for "Fake razes" is that it isn't a fake out to the bot, it's a technique to lower the casting time to zone out the opponent/hit faster if the opponent chooses to dives. Seems like a pretty easy trick to learn.

Creep blocking is just learning "positioning myself here (in front of the creeps) tends to improve my chances of winning", where "here" isn't far from where it would be positioning itself without creep blocking. It's a minor adjustment to the behavior with a substantial effect on the game outcome, of course it learned it.

In this post, a member of the OpenAI team characterizes this form of coached self-play as being supervised. If you are assigning a value to an action that is not an explicit goal of the game and is not a consequence of violating the game's rules, it would seem to fit the definition of 'supervised'.

https://news.ycombinator.com/item?id=15002150

What hero do you think would be a better test?

Techies.

It's not about which hero 1v1 is not a competitive part of dota, if you claim beat pro players it has to be 5v5 with banpick

Otherwise just say it's 1v1 mid...

Sure, but it hadn't been done before. They say they'll go for 5v5 next year, and if it wins I'm sure we'll move the goal posts further again.

I'll precommit to being very impressed if a bot can achieve >5k MMR in solo ranked or if a bot-controlled team wins a tier 6 (or higher) battle cup.

When Checkers and Chess and Go pros were beaten by AI at the actual game, people were impressed, because it was impressive. (The goalpost moving involved claiming each consecutive game could not be solved.)

I don't think anyone here is saying Dota 2 won't be solved eventually or that its complexity is beyond the realm of AI (as was claimed of Chess/Go in the past). They're just saying this particular achievement isn't actually meaningful progress. It's using known techniques to do something those techniques are known to do.

> I'll precommit to being very impressed if a bot can achieve >5k MMR in solo ranked or if a bot-controlled team wins a tier 6 (or higher) battle cup.

Me too but it will never happen. Granted I haven't played DOTA but I've played many other competitive mulitplayer games and they all require one thing which bots currently lack: Communication. A bot playing the entire 5 man team though, that's a different story!

Using four skills in one hero is similar to using 20 skills across 5 heroes. It's not communication, just an extension of a single bot. (It's like a single player micro'ing all 5 bots.)

Surely the different bots would communicate via RPCs or some other API. It wouldn't be much different from 1 single bot, especially if 1 bot decided to coordinate everything and the other 4 bots decided to just follow orders.

Either, (a) high risk/reward heroes: Pudge, Huskar, maybe Weaver or Puck, or (b) highly complex heroes: Invoker, Morphling, Earth Spirit, Nature's Prophet, Techies, maybe Meepo.

Would love to see courier snipe from lv 1 by NP or even man fight in the base

i would love to see 2 meepo bots go head to head mid

I know it'd need a much larger training set, but honestly you're not playing DOTA unless the full hero pool is in play.

Never mind 5v5, a 1v1 bot that would let Dendi/Sumail/RTZ etc choose any hero and still win would be much more impressive.

And when that's done, 5v5 will be much more impressive, so we none the goal posts.

Reply on news.ycombinator.com