Hacker News Reader

Comment by kibwen | Hacker News Reader

In the interest of a "fairer" comparison, I wonder how much of a difference it would make to force the AI to simulate mouse/keyboard input and interpret the raw screen buffer output, rather than using direct APIs into the game's guts, to more faithfully emulate its human opponent. I'm guessing the peripheral inputs wouldn't be much of a hurdle, but the image processing step could be very interesting.

View on HN

How do you know it's not doing that already? I was under the assumption that open ai agents interacted with the environments through vnc.

DOTA has a bot API (https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting)

If you're interested I did something similar to what you describe for MarioKart http://kevinhughes.ca/blog/tensor-kart. Its a lot simpler than DOTA though

Raw screen is not as big of a deal as it would seem, in DotA and Starcraft you can glimpse most of the information you can see on a screen through the minimap.

It would seem to me that this is an oversimplification. Doing screen based video processing is not simple and is not exact especially where camera movements are independent of game dynamics as well as the difficulties inherent in doing fast frame classification- there has been some good work in this direction recently but far from perfect(yolo, ssd). The route deep mind took in partnering with the game manufacturers to gain inputs directly while receiving global information about game state seems the simplest for their goal of training rl agents. We considered doing a project around this but figured it was a massive iceberg.

Oh I misread the "Direct access to game API section", I thought they meant making it so that the machine can only see "what's on their screen", so if someone were to cast a long range skill for instance they wouldn't get that information. Which is a distinct advantage for machines.

That is indeed, by no means trivial. I do think it's pointless though, that's just another exercise for the sake of making it another exercise.

Reply on news.ycombinator.com