Hacker News Reader

Comment by karpathy | Hacker News Reader

I remember that they had an impressive demo at the NIPS Deep Learning Workshop in December (probably a highlight for me and a few people I talked to afterwards) of a reinforcement learning agent playing Atari 2600 Games where the input consisted of an image that was fed through a Convolutional Neural Network. There is a paper on Arxiv that describes the approach ("Playing Atari with Deep Reinforcement Learning"[0]) but unfortunately I don't think they made the videos available.

Deep Mind has a whole bunch of talented and serious people so this is an exciting acquisition.

[0] http://arxiv.org/pdf/1312.5602v1.pdf

View on HN

I actually read this paper a couple weeks ago as part of a deep learning reading group that I co-run. While several of these authors are household names in the RL community, this paper was not actually that impressive to me.

The only real "deep" learning here is that they used a GPU library and stochastic gradient descent to perform Q-learning updates on a network with 3 large hidden layers. It was an interesting application paper, but I suspect that the Google acquisition is for something more novel than this work.

> surpasses a human expert on three of them

I hazard it's not very impressed with your Space Invader score, either.

I suspect google wants to port this to their D-wav, probably sell API access to it. Rent an AI

Compare http://wavewatching.net/2014/01/18/scott-aaronson-again-resi... and (more directly) http://www.scottaaronson.com/blog/?p=1643

This deep learning group you mentioned. May anyone join in?

Sorry, it's for UT Austin PhD students only.

We worked on a similar experiment two years ago (deep learning + reinforcement learning algorithm + some innovations, to learn to play Atari 2600 games). We obtained similar scores in the games we tested but we did not submit any paper because we considered that the scores were not good enough. In particular, for Space Invaders, you can easily get 600 points by hiding behind a shelter while continuously firing, and never learn how to avoid the bullet.

So, I was not impressed by their results on Space Invaders.

Overall, we struggled to learn long-term strategies (finding pure reactive strategies is easy) and to learn to avoid bullets. They did too: "The games Q*bert, Seaquest, Space Invaders, on which we are far from human performance, are more challenging because they require the network to find a strategy that extends over long time scales."

=> that's the real challenge...

And what's interesting is they seem to be hiring a whole lot more [1]. If they are hiring a "growth hacker" they must be close to having something that's out in people's hands.

[1] http://workinstartups.com/job-board/jobs-at/deepmind-technol...

Reply on news.ycombinator.com