From Scratch: AI Balancing Act in 50 Lines of Python | Hacker News Reader

From Scratch: AI Balancing Act in 50 Lines of Python

towardsdatascience.com

67 pointsamasad9 comments

Initializing the policy to random and performing thousands of trials is just about the simplest implementation of "AI" you could possibly ask for.

It's nice to see someone actually going for the MVP.

The mind blowing thing is that it actually works, and with rand(0,1), it'll get you results with only 10 guesses (though it's a bit cheating haha).

Fun fact: a policy of [1,2,3,4] will solve the game.

When I started off the article, I was thinking of doing some kind of evolutionary-based policy search but turns out none of that is even necessary.

I hope to have a more technical follow up w/ deeper analysis and hopefully another one that goes into harder games where random guessing is okay, but there are more "sample efficient" methods.

Yeah I've been meaning to work my way up the complexity ladder on openai's gym.

Seeing that it can be reasonably done by a random walk and the fact that it's only 4 dimensions kind of makes me want to do a uniform convering of the space and visualize the results.

You should! I did some analysis myself a bit, have to dust it off to make it bit more presentable. There are definitely clear boundaries of "good" policies.

If you do decide to do an evolutionary-based policy, I highly recommend messing around with NEAT (https://github.com/CodeReclaimers/neat-python).. I have successfully used it to play a number of SNES/Genesis games.

I even made a tutorial series. https://www.youtube.com/watch?v=pClGmU1JEsM&list=PLTWFMbPFsv...

Apologies for the shameless plug.

Not sure why this has to be AI specific.. Terry Davis was doing this 15+ years ago: https://youtu.be/25hLohVZdME

Seems to be more of an issue of knowing calculus and control system patterns.

It's not. It's a trivial problem to have something easy to do when starting out with reinforcement learning.

Discussing "state":

> In the game of pong, it might be the vertical position of each paddle and the x, y coordinate of the ball.

The velocity vector of the ball is also a (very important!) part of the state.

Good point! Edited the post :)

(Admittedly I've never built anything to play pong )

Reply on news.ycombinator.com