Deep Reinforcement Learning to Play Space Invaders [pdf] | Hacker News Reader

Deep Reinforcement Learning to Play Space Invaders [pdf]

nihit.github.io

56 pointsKatydid11 comments

So... the original DQN paper already did Space Invaders as part of the ALE environment, getting better than their human player's score. The DeepMind source code as well as other reimplementations have been out for years, and likewise for Double Q-learning, so it didn't really need someone to reimplement it either. Looking at this paper, I'm not sure what is new or interesting, other than the experiment with using the Atari RAM as features (but even that doesn't seem as interesting as the paper which used MCTS on the RAM: "Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning", Guo et al 2014 http://papers.nips.cc/paper/5421-deep-learning-for-real-time... ). Or is this just the report from an (impressive) student project and not intended to be groundbreaking research?

It seems like a student project, not groundbreaking research.

Just a nitpick, but the original didn't do better than human scores. It does better than their human panel, who seem to suck at space invaders and a whole bunch of other games. In general DQN does worse than expected at space invaders for some reason (compared to, say, galaxian, a very similar game, at which it does very well)

PLAGIARISM

Third paragraph of Intro in original DeepMind paper https://arxiv.org/pdf/1312.5602.pdf

Second paragraph of Intro(5th line) of this project. http://nihit.github.io/resources/spaceinvaders.pdf

They didn't even bother to do something original except for using the RAM state instead of pixel values like the original paper. It's something that's easily doable with the openAI gym. This seems like a course project at stanford.

For convenience.

Original:

However reinforcement learning presents several challenges from a deep learning perspective. Firstly, most successful deep learning applications to date have required large amounts of hand-labelled training data. RL algorithms, on the other hand, must be able to learn from a scalar reward signal that is frequently sparse, noisy and delayed.

New:

However reinforcement learning presents challenges from a deep learning perspective - most supervised deep learning applications require large amounts of labeled training data whereas reinforcement learning algorithms must be able to learn from scalar rewards that are often delayed.

Immediately after the plagiarized section is a citation to the paper which was plagiarized (Mnih et al.).

Hi, one of the authors here. Yes this is a school project. This pdf was a draft, not the final version for the class submission (the paragraph in question never made it to the final version). Don't know who shared the link here, but it is updated now.

Off topic but related to deep learning. Did anybody encounter use of machine learning in mesh generation, as in 2D, 3D triangular, quad, hex mesh generation?

I am also interested in this. I've seen work done in generative models for voxels but nothing in meshes unfortunately. I also haven't looked quite that hard.

Yeah, it seems like deep learning would be apt here, but couldn't find almost anything. Time for disruption :)

At a glance, this looks essentially like a recreation of Deepmind's work from a couple of years ago. Is there something new here I'm missing?

I wonder if this could ever learn to play Spelunky? :-)

Reply on news.ycombinator.com