Teaching Your Computer To Play Super Mario Bros | Hacker News Reader

Teaching Your Computer To Play Super Mario Bros

ehrenbrav.com

122 pointshivebrain24 comments

I believe that mario bros is deterministic and doesn't involve any randomness. Did you implement random start for fair evaluations to make sure that he found patterns instead of learning what to do by heart (which is a lot simpler). Random evaluations is a chain of random input at the start to randomize the environment a bit.

No - I actually didn't use a random start. But when Mario dies far enough along in a level, he starts from near that point rather than from all the way back at the beginning, which kind of serves that purpose. I agree that what's most interesting is truly learning to generalize rather than just learning a sequence of moves that happen to work...

Well not really since the fact that he dies and respawn at that point is still deterministic. Cool project though

If you introduce artificial randomness your not teaching it to play Supper Mario. One option might be to include randomly pressing the wrong button with some low probability.

I think that's the epsilon parameter. That's the chance that, instead of playing the move Mario thinks has the highest value, he plays a random one instead. That in theory should help prevent him from falling into a local minimum. This probability starts at 100% at the beginning of training and over the course of 1 million steps, ramps down to 1%

This might suggest it has learned by heart, not by pattern:

In the example video, during the three lives on the second level, it does worse on the same parts of later lives.

Of course, to draw any conclusions, we'd need a lot more than one run.

I wonder if you were to train a single NN on multiple games (say maybe Mario, Commander Keen, Duke Nukem 2, Battle Toads, etc) if it would perform better against any one game than say a NN trained against just the one same game? I know I have gotten better on any particular game when I have played more games of the genre. If that;s the case could we use this approach to make a General Game Playing AI.

The original DQN was created to do that.

This is a similar project written in Lua, with a different Mario game: https://www.youtube.com/watch?v=qv6UVOQ0F44

Referenced in the article as one of the two predecessors to this project.

Neat! I also did a similar project but haven't invested the time in letting it run for very long. 4 million frames is about 37 game play hours, right? What were your results? Can Mario clear the first level?

When I implemented this, I scored the q-learning using real score and live count. I didn't need to add in a bonus for running right because it would randomly grab the first coin or randomly jump on the goomba quite frequently.

Also, I implemented mine using tensorflow, so it's possible I just sidestepped some bug.

The video was made after 4M training steps.

At that level, he routinely clears the first level and makes it about halfway through the second.

The improvement was still going strong at that point, but I ran out of patience. I'm doing another run right now...

if you check the video you can see it clear the first level and fail a couple of times on the second one

Fail a couple of times is generous, it get's a game over around half way through the second level.

I've been collecting these videos. So far I have Mario, an RC car, Montezuma's Revenge, and a NLP for grammar.

https://www.reddit.com/r/watchmachineslearn

This was a great explanation. Thank you. I wonder if similar technique can be used for lane following robot that would understand stop signs, traffic lights and avoid collisions.

I've been thinking of creating a simulator game for driving, however have absolutely no idea how to program the AI side of things.

Yeah, you'd need to "punish" it somehow for running stop signs and lights :)

If there's a good driving sim game on Nintendo, it should in theory take minimal modification of my existing code to make it work...

I really wish someone could try deep neural networks on my favorite mario doujin game that combines mario with portal: http://stabyourself.net/mari0/

Also, check out https://www.youtube.com/watch?v=qv6UVOQ0F44 for another cool ML-Mario implementation.

How long did the training take? I am still training the default breakout game and it still has not learnt yet. I have been training it for a week on a 4 CPU machine. Should I go for GPUs?

Definitely use a GPU if you can - it's just so much faster. You could also try training it on a powerful machine in the cloud if you don't mind spending the money for it...

I'm running a 8 core machine, 12+ GB RAM, and a 980 GPU. It takes roughly 24 hours for 1 million steps in Mario...

Sorry for content-free comment, but this is really interesting, and I wish it had gotten more discussion. Starred and saved for later.

Glad you think so! It was a lot of fun to put together and I'm contemplating working on something like Zelda next now that I have a computer-controllable NES emulator working...

missing the best video about teaching computers to play video games: https://www.youtube.com/watch?v=xOCurBYI_gY

Reply on news.ycombinator.com