This book (by two of the same authors) is also good:
Just my 2 cents but I find PCI a very shallow book without much "deep technology" in it either.
AI cannot be divorced from the underlying math. PCI takes (in my opinion, feel free to differ) a math-lite, "dummies guide" approach to AI algorithms. I realize that my opinion is in the minority, and a lot of people think that the book is very cool. So take it with a grain of salt.
I loved PCI because it was math-lite.
I'm a business applications programmer by trade and fun stuff like this isn't anywhere on our radar. I also have a family and the associated obligations like T-Ball and dance recitals so I don't have the time to dig deeply into the algorithms and math. PCI was a good balance to me between showing the theory and also providing concrete working code.
My only quibble with the book was that a lot of the code was not idiomatic Python. (un-Pythonic, if you will.)
Just my $0.02.
That said I have three complaints:
1 I think it tries to stay too high level on some areas where it is truly unavoidable and wouldn't scare anyone off to add a little depth.
2 I think it too often labels their implementation with a name that is associated with a family of techniques. This is simply going to embarrass/mislead the reader. In the interest of maintaining simplicity the book states several things that are flat out wrong.
3 No references. If someone wants to take the next step there is zero guidance. This is important because someone learning about a deep topic could read the book and a) think they understand it (not likely) and b) not have a clue where to look next.
"[t]he algorithm takes the difference between every pair of items and tries to make a chart in which the distances between the items match those differences"
This is referring to multidimensional scaling which is a large family of techniques, some of which attempt to achieve this goal (delta ~= d), but not necessarily. Granted, in this case the algorithm presented DOES attempt to do this. BUT, the algorithm presented is not "multidimensional scaling". It is an implementation (not even a good one) attempting to minimize one of many possible objective functions.
I noticed several more when I flipped through it initially, but this is the type of thing I'm referring to.
On the other hand going right into code examples is useful, including jumping right into getting real data downloaded and worked on.
I also do not have PRML, but Neural Networks for Pattern Recognition by Bishop is excellent (and includes many non-NN related items).
ESL is excellent and, to me, the best modern text in machine learning. It covers many of the topics in PCI both at a reasonable level and in much more depth and provides MANY references (100s?) to dig deeper.
If that's all you get out of it great, but there's a lot of interesting stuff in there, covered in enough detail to get you started, which is more than enough to make me happy.
That is exactly what it means. "Deep math" is not learned because people have nothing else to do with their time. The moment you try to apply or extend the techniques in PCI beyond the toy examples in the book you will see the need for the "deep math".
Just one example. BackPropogation Neural Networks are covered very superficially PCI. To get a real understanding of BPNNs, read Chris Bishop's Neural Networks for Patttern Recognition. That's a whole book on one type of neural network.
I quote Peter Norvig's Amazon Review of Bishop's book. (emphasis mine)
"To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is palstered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere, but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef."
"If you want understanding of the techniques, you have to understand the math". Without understanding you can't (a) judge the appropriateness of a particular algorithm to a data set (b) know which variant of the algorithm to apply (c) understand the results, especially when they don't make sense (d) debug the implementation if required. In short at best you are making calls into a black box library and praying the results make sense.
PCI skips the "understanding" part of AI aIgorithms (which does need math as Norvig points out. Most AI is applied math)and provides a superficial outline and many blackboxes. E.g: the "use libSVM for Support Vector Machines" idea PCI propagates (after a very superficial overview of SVMs) .
LibSVM is a great library,but trying to use it to "SVM ize" your code without understanding exactly how exactly SVMs work (which does need "deep math" ;-) ) , will give you ... ummm.. suboptimal ... results.
One of my friends tried to use algorithms from PCI to add "intelligence" to some code for his startup (he first ported the PCI code to Ruby) and ended up getting nonsensical results. Eventually I looked at the data, threw away his code and replaced it with a different algorithm which was appropriate. But, hey, I got some equity in return for the AI code so that turned out all right :-P
All that said, I have no problems with folks using/liking PCI. I was just reacting to the "deep tech" phrase in the original post.
1) I mentioned multidimensional scaling elsewhere. In the book there is implementation and it doesn't even say what the goal is (in terms of objective function) and/or how well it performs. There are many different goals of multidimensional scaling. If one understand even the objective function they are minimizing then the algorithm to do so will make more sense.
2) The book grazes over SVM's and makes a few blanket statements that I haven't heard from serious researchers. E.g., "[t]he one that is usually recommended ... is called the radial-basis function. The radial-basis function is like the dot-product in that it takes two vectors and returns a value. Unlike the dot-product, it is not linear and can thus map more complex spaces." Most of the kernel/svm folks I know would never be so confident in saying such a blanket statement as it entirely relies on the data/situation.
It doesn't come with pre-canned python, but honestly almost everything (in terms of code) in PCI is available somewhere on the web and/or already built into python libraries, matlab, and/or R.
I think I got a lot right when I set the level but having seen it in the wild there are things I would have done differently too.
Mostly I wanted people to get excited and see that they could do this right now and that there were good reasons to. I never intended to compete with Norvig, more to create something that would inspire them to want to read it.
1) The examples are real python code, and many chapters the code samples are small enough that you can follow the text by typing the code into an interpreter to try it out
2) He doesn't just test algorithms on standard datasets like MNIST or MovieLens - He gives programming examples for how to extract your own datasets e.g. by showing how to use the eBay api to download prices from eBay, or how to use the facebook api.
To each his own - I thought this book could easly be condensed into a list of mathematical theorems - a page worth of links to wikipedia.
1) The examples are real python code
Really crappy python code... definitely not the kind of code one should be looked at as example.
He doesn't just test algorithms on standard datasets like MNIST or MovieLens - He gives programming examples for how to extract your own datasets e.g. by showing how to use the eBay api to download prices from eBay, or how to use the facebook api.
He combines math theory with code and API's? What's the point of that?
If you want to learn how to code python, read a python book. If you want to learn math, read a math book. If you want to learn API's, read the API docs. If you want to learn all three, read all three, but don't think that you know any of them if you see a bunch of badly coded examples of the results of knowing them.
I think that any maturing field needs a hierarchy of explanations that go progressively deeper into the subject - the explanations of the second law of thermodynamics that you get in freshman chemistry course are superficial compared to what you would deal with in a graduate course, but this doesn't mean that chem101 should be taught at the grad level, or that it is pointless to teach freshmen about the second law. The real danger is if you lie or misrepresent the facts in an attempt to simplify. I'm sure there are some errors and falsehoods in PCI but I didn't see anything too bad in the one day that I've had the book.
Learning API's by reading the docs sounds like trying to learn english by reading the dictionary - It's just my personal preference, but I feel that I learn more effectively by reading real source code that works (even if it is horribly written)