He (Andrej Karpathy) has got a series on youtube which goes through how he made it!

https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...