Throwing in my personal goto reference on gradient descent : http://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradi... "An Introduction to the Conjugate Gradient Method Without the Agonizing Pain" - Jonathan Richard Shewchuk.