Hacker News Reader

Comment by spectramax | Hacker News Reader

Ian Goodfellow’s Deep Learning book pretty much useless. I own it and have read through most parts of it. I couldn’t explain it better than top Amazon reviews:

https://www.amazon.com/Deep-Learning-Adaptive-Computation-Ma...

And I’m surprised to not find Aurelion Geron’s absolute masterpiece listed below. I believe it is the best machine learning book ever, although Statistical Learning mentioned in the article is really good as well :

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

View on HN

For what it's worth, I disagree quite strongly with that review. The book is aimed at those with a pretty mature appetite for abstract mathematical reasoning, but not much specific knowledge in the areas of statistics, machine learning, and neural networks. It's an actual graduate-level book, and one must approach it with the appropriate background and education.

The Goodfellow book is not complete as an academic intro, but no one book can be. It's not very useful as a practical tutorial, but no book seeking this could cover the mathematical arguments that Goodfellow's book does. I found Goodfellow's book extremely useful for consolidating a lot of handwaving that I'd seen elsewhere and putting it in a slightly more rigorous framework that I could make sense of and immediately work with as a (former) mathematician.

Goodfellow's treatment is especially useful for mathematicians and mathematically-trained practitioners who nevertheless lack a background in advanced statistics. The Elements of Statistical Learning, for instance, is extremely heavy on statistics-specific jargon, and I personally found it far more difficult to extract useful insights from that book than I did from Goodfellow's.

The problem with Goodfellow's book is that it is half-baked. I don't know why he has to introduce Linear Algebra section for a 3rd of the book (which ends abruptly) but then moves on to the ANNs. If Goodfellow intended this book for mathematicians, that whole section about LA can be omitted with literally no loss in the book's value. The whole book arguably feels rushed.

So, no amount of praise and mathematician's justification makes sense. I agree that it is inclined for mathematicians, but this book is overrated and it is terribly due for a rewrite, update and frankly in my personal view - the writing style.

I am curious of specific parts of the book you found valuable.

The section on linear algebra moves quickly. I don't see how a book including extra prerequisite material is an example of it being half-baked or rushed. Surely that would actually be an example of the opposite? That section is a nice reference to have immediately available for things like SVD and PCA.

Perhaps restructuring the LA part into an appendix would be preferable. For mature readers, it nevertheless serves as a way to focus and agree on notation that is used for the rest of the book.

The best textbooks, the ones considered classic gems in their field, are careful about what they say and what they leave out. You get the sense that every word is in its right place.

By comparison, Goodfellow and his co-authors seemed to just dump everything they know onto the page. It's fragmented, bloated, and it meanders all over the place. Goodfellow was on a recent podcast where he seems to acknowledge that the book straddles an awkward place between tutorial and reference.

I don't mean to sound too harsh. I appreciate its scope, and I've certainly read much worse textbooks.

Considering how much is not in the book -- and that the book is not even that large as far as textbooks go -- part of me feels this criticism is somewhat disingenuous. I'd agree that the direction of the final part on research topics feels fragmented, but the rest of the book certainly doesn't. It's very clearly focused on developing deep neural networks and doesn't actually meander at all.

If the worst you can say is it's not a classic text, that's really not saying much at all. I feel weird defending the book so much when to me it's just a book I found useful and I don't even feel that strongly towards it. But the strength of some of the criticism here doesn't seem motivated by the book itself.

This is pretty harsh. I use Goodfellow as a reference text and then supplement the mathematics behind it with more comprehensive texts like Hastie or Wasserman. Maybe if you sit down and read it cover-to-cover, it will seem disjointed. But I usually read chapters independently - I recently read the convolutional neural network chapter in preparation for an interview and I thought it was fine.

I much prefer reading Karpathy's notes and watching Stanford CS230 than to delve into Goodfellow. It sits on my shelf collecting dust.

So it's a reference, not a pedagogical tool then?

A reference implies you already know the topic and just want an index to jog your memory for things you can't hold all in your head at once.

That is different than a pedagogical tool. If so, you shouldn't recommend it to those want to learn the topic.

Phrased like this, I agree with you. I didn't think about it like that.

I agree with the poster below. Outside of classes, lecture notes, the books I listed, and Sutton/Barto (Intro. to Reinforcement Learning) have taught me the material. I use Goodfellow to brush up before interviews or jog my memory about topic I don't work with very often (like computer vision).

What are Wasserman and Hastie?

- Wasserman has a book called "All of statistics" that gives a lot of the background required to understand modern machine learning

- Hastie is a co-author of two machine learning books, one is "Elements of Statistical Learning" which is very comprehensive, and "Introduction to Statistical Learning", which is more approachable by people without too much background in stats.

FYI, there's a new edition of Geron's book coming out in August which will include Keras: https://www.goodreads.com/book/show/40363665-hands-on-machin...

I think the Amazon review is rather dramatic, and probably not in a position to comment on style. I thought both Goodfellow and Geron were good. Goodfellow is deep learning for academics coming from a different field; Geron is deep learning for software engineers.

That's the thing, even as an academic book it falls short. It feels disjointed, unorganized and poorly written - and most frustratingly, incomplete.

The reason for Goodfellow's popularity is that it was publish in 2014 right at the turn of exponential interest in Deep Learning after AlexNet. It took off and became popular, but readers now feel it is stale for the aforementioned reasons.

I certainly agree it could be better, but I also think "disjointed, unorganized and poorly written - and most frustratingly, incomplete" more or less sums up the field of deep learning :).

I can second the recommendation for Geron's book, it's absolutely stellar.

Do you think there is a need for a better written DL textbook? I definitely agreed with the review you linked.

I've always thought that Hands on ML by Geron was great implementation wise, but lacking in the mathematical rigor and depth. While I would have a general sense of what is going on after reading it, and I'd certainly be able to structure and implement a model, I don't know if I would have any deep intuitions.

Geron's book is more of a tutorial/cookbook coalesced with important insights into the practice of machine learning. So, I recommend reading Introduction to Statistical Learning (and Elements of Statistical Learning for theoretical background) before jumping into Geron's book. As engineers, I agree we need to have some theoretical background but at the same time, we are applying this knowledge to real world problems. Geron's book is invaluable and I hope publishes more, it is a gem.

2nd edition coming in August. Preorders opened a few days ago. He posted on twitter. Some preview chapters available on O'Reilly's site.

I've heard poor things about Goodwell's Deep Learning book as well, what is a good alternative?

Take CS230, I believe it used to be taught by Fei Fei followed by Karpathy and I don't know who teaches it now.

For what it's worth, I read Goodwell's book cover to cover and loved it. It answers "why" questions rather than "how" questions, but those are the questions I had, and you can find "how" questions answered for your framework of choice on the internet.

Gérons book is both entertaining and educational, I really enjoy it so far.

Reply on news.ycombinator.com