Hacker News Reader

Comment by Groxx | Hacker News Reader

I've read a good 1/4 of Artificial Intelligence - A Modern Approach, and scanned a handful of pages further on, and thus far it's been excellent.

So far, it looks to be very much rule / first-order logic oriented, but that makes sense, as it's been the most well understood, and much more easily provable / understandable once something is built. There's very little about neural networks, for instance, or genetic algorithms, aside from an explanatory section or so. Though I may have the first edition, and apparently the 3rd came out last year.

So, if anyone is interested in a relatively-introductory, extremely readable book on machine learning, it's the best I've come across so far.

View on HN

> There's very little about neural networks, for instance, or genetic algorithms

For good reason. Outside of neuroscience, there's very little reason to use neural networks for classification or regression problems, and outside very ill-posed optimization problems, there's very little reason to use genetic algorithms. If you want a good MODERN machine learning book (which is where your post seems to imply your interests lie), try the Elements of Statistical Machine Learning by Tibshirani, Hastie and (I forgot the third guy's name, sorry). Anyway, it's included in the posted link.

Not sure neural net expert Yann LeCun (yann.lecun.com) would agree with your neural networks comment. Neural nets have taken on a life of their own since the time they were originated and actually have little direct connection with neuroscience these days.

This is true, but the very good deep learning results are very recent (from less than 5 years ago), and they generally require very different models and/or training strategies from the standard multilayer perceptron you find in most AI textbooks. So it's an omission, but I'm not aware of any textbooks that cover these new neural networks in depth, as basic research in this area is still going in a frantic pace.

Yeah, I'm finding that there's little book-format info on deep neural networks as well. Similarly for belief networks. Gotta hunt down some journals, and wield some Google-fu.

The most promising approaches for neural nets these days involve learning generative models using Restricted Boltzmann Machines (RBMs). Geoffrey Hinton at UToronto has some good seminal papers on these, like http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf. See http://www.cs.toronto.edu/~hinton/ for excellent videos of digit generation / classification.

As a note, this approach outperforms all others (sans tweaks), whereas neural nets used to be beaten by SVMs and the like.

Actually, the best recent results in this area involve discriminatively trained stacked auto-encoders using second-order algorithms (for which the code, as of Sept 2010, is still private to the group at the university of toronto), as described in James Martens's paper at this year's ICML: http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pd...

This is a very popular undergraduate book which I hated. I simply hated how the work is represented - instead of having good mathematical examples (and a clear notation!), they have a lot of text and a bad/substandard notation.

Then this is probably right up your alley: http://research.microsoft.com/en-us/um/people/cmbishop/prml/

"Pattern Recognition and Machine Learning". And I dare say there may be more space used on the equations than on the text.

Reply on news.ycombinator.com