Hacker News Reader

Comment by wjnc | Hacker News Reader

Been staring in awe at this with my little sons (2&3). We actually get enthusiastic when the first signs of understanding arise. Where can I begin learning about this? I've done several sklearn examples, but this application to 'a new world' (like a racing game), how do you build that?

With build I mean, how do you structure inputs and objects so that the goal is met? The goal for racing is both steering, braking, trottle...

View on HN

I did something very similar to this for an artificial life class back in college. My nets took significantly longer to learn how to race for some reason, but in the end it worked out. What I did was start off as simple as possible, just using a circular track so that it's easy to test bounds on the inputs (position greater than inner diameter and less than outer). My nets would get their sensor inputs at each time step, and would output a steering angle and speed. The speed would basically just be directly added to their current position after pointing them in the chosen direction. The fitness function amounted to seeing which racers got around the track the fastest.

You can get some pretty interesting behavior if you start with a barebones model of the world like that. At one point my racers started teleporting around parts of the track by taking advantage of the fact that I wasn't doing continuous collision detection - they would detect a turn and then output a speed and direction that would cause them to instantly appear on the other side of the corner without navigating the curve on the way. It was interesting to watch them balance the ability to teleport against the range of their sensors.

This sort of thing really lends itself to incremental tinkering, especially with javascript as powerful as it is today for prototyping. That gives a lot of intermediate feedback that will give you chances for finding new strange things to investigate. Adding features like tricky track generation is a decision you can make along the way if it seems interesting.

It's a genetic algorithm, which means that it has a fitness function for how close the goal is to being met (choosing the furthest traveled car), and a 'gene', which is a sequence of instructions. The game utilizes a Neural Network to classify when it should turn/which way it should turn/how it should steer/etc, and then this NN is adapted into the genetic algorithm, which effectively generates random genes, follows each of those genes, then chooses the best 'fit' one according to the fitness function. Next, the algorithm restarts with the exception of now it's randomness is slightly based on the best fit gene from the last epoch, and repeat until you've got a gene that works.

Imagine having a wall, and a certain portion of it is sticky (since it's covered in invisible glue), but you don't know which portion, and you certainly don't want to touch it to find out. So you throw spaghetti at the wall, and see which portions stick. Next, you throw more spaghetti at the part that seems to stick, and repeat until you've got the entire sticky portion mapped out.

There's an old genetic algorithms demo with smart rockets and path-finding here: http://www.blprnt.com/smartrockets/ [warning: requires flash].

If you're actually interested in building one of these, Daniel Shiffman wrote The Nature of Code: http://natureofcode.com/, which is an introductory level book for this kind of thing. He uses Processing in the book itself for modeling, but he also goes over the theories pretty well.

If you want some formal literature, I took a brain and behavioral processing class in college, and the professor still has his slides up here: http://www.life.illinois.edu/mcb/419/sessions/index.php, along with links to papers and homework scaffolding.

From what I can see, the course you mentioned looks very cool, however it seems a lot of the slides and linked papers are password protected. Do you have copies mirrored and/or know if the professor would be OK with these being shared?

(Don't worry if not, As much as I'd love to browse some of whats on there, I don't want to make extra work for you or get anyone in trouble!)

Hmm, that's strange, I can still access the files despite being in incognito mode on Chrome. I'm fairly certain the professor doesn't mind this being spread around, but I don't know HN's policies on sharing stuff like this, so shoot me an email (it's in my profile) and we can discuss it.

Reply on news.ycombinator.com