Hacker News Reader

Comment by BaronSamedi | Hacker News Reader

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.

View on HN

If your goal is to build General AI, getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as experience playing past games. I have yet to see any video game bot make use of this stuff to figure out how to play Zelda, for example.

An AI which can learn any game from "looking" at a screen is a very ambitious goal. I doubt it is achievable in the near term. Humans can do it because of years of learning about the world. Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games such as Starcraft, LoL, and Dota--especially the latter 2 since they are team games.

That's not their goal though.They want AGI.

Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games

Those sorts of goals are better suited to individual hobbyists. OpenAI is a blue sky research project set up by billionaires with the goal of improving all of mankind. I hope I'm not being too uncharitable when I say that your comment reminds me of those who scorned the Apollo program for its ambition.

I support research in AI of all kinds, however, its history is one of vastly over-promising and under-delivering. Modest but demonstrable progress is better, in my view, than yet more hype.

Seems unlikely. The focus seems to be on improving AI through "vision". The idea is to make the AI learn skills the same way a human would (at least in the first years of life). Google's AlphaGo also learned from screen pixels.

So these would be human-like bots, rather than bot-like bots, like you normally have in games. The bot would simply learn by doing, until it masters the game, not by getting access to game algorithms.

> Google's AlphaGo also learned from screen pixels.

Source? That literally seems to make zero sense to me. Go can be represented in a super-simple state. Why make it spend millions of cycles learning to categorize pixels into that state you already have?

I would guess that they trained AlphaGo from many thousands of hours of match footage. Writing a computer vision script to segment / extract the data may cost cycles as you say, but would save many human hours by eliminating the need to re-watch the footage and literally type out state information for each move.

Alphago was actually trained directly on game state (plus some extra computed state like "how many liberties will I have if I play this move" or "will I win this ladder"). A huge number of pro games (and countless amateur games) are available on servers like KGS in a nice computer-digestible format.

I'm not too familiar with Go, but in chess we have millions of text-format games and very little real-time footage.

Again, that doesn't make sense because the moves have almost certainly already been typed out by somebody. It's the same in chess. There are databases containing millions of games.

> Google's AlphaGo also learned from screen pixels.

It did not. It received the state of the board as one array, another board state array for capture/komi (since Go does have global state which is not visible purely from the board representation), and a few additional features to help it out with stuff like ladders. It was architected with convolutional layers, but over the Go grid, not pixels. See the AlphaGo paper pg11 for the exact structure of the input: http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e...

Could it have learned from pixels (augmented by the additional necessary global state)? Sure. But that would've been a waste of computation since the visual layout of a Go board is fixed and static, unlike Atari games.

Getting the game state data means deciding a-priori what features the AI should learn on. The whole point of the deep learning paradigm is to allow a machine to learn such features that enable good prediction, visualization, generation (aka. hallucination), etc.

Instead, researchers have provided the raw feed input data to these agents with the hope that the learned features could be interpreted as game state data by humans.

I would say that it is a point of deep learning rather than "the whole point". For an AI to interact with the real-world then building models from vision (as we do) makes a lot of sense. In the virtual world, however, it makes no sense. Model data is already available and the AI has no need for something as inefficient as vision. We humans have to use vision (and sound, etc) in games because we have do not have access to direct data feeds, computers have no such limitation. Why cripple the AI by imposing human limitations on it?

If they want their AGI to applicable to the real world or software with incomplete or insufficient APIS they have to do it the way they are doing it here.

There isn't an API for me to check if i'm still on the foot path and not the road as i walk down the street.

I can't use an API to tell me water is boiling and that i shouldn't stick my hand in it.

If the goal is to have AI that is aware of the real world (even a chatbot), then using game state is a crutch that doesn't help us solve the real problem.

Reply on news.ycombinator.com