Hacker News Reader

Comment by f0e4c2f7 | Hacker News Reader

>I know Sheldon; he believes it's a crime to teach people determinants.

Any way to explain to a lay person why?

The book is open access (https://linear.axler.net/), I am going to quote directly the author in the preface:

<< all linear algebra books use determinants to prove that every linear operator on a finite-dimensional complex vector space has an eigenvalue. Determinants are difficult, nonintuitive, and often defined without motivation. To prove the theorem about existence of eigenvalues on complex vector spaces, most books must define determinants, prove that a linear map is not invertible if and only if its determinant equals 0, and then define the characteristic polynomial. This tortuous (torturous?) path gives students little feeling for why eigenvalues exist. In contrast, the simple determinant-free proofs presented here (for example, see 5.21) offer more insight. Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra— understanding the structure of linear operators.>>

If you like mathematics, it is actually a pretty nice book.

The philosophy of LADR is described in his paper Down with determinants: https://www.axler.net/DwD.html. In short, he thinks they obscure proofs. I love the book, but AFAIK only the compactified version (excluding all proofs, examples, and exercises, along with most comments) is open access: https://linear.axler.net/LinearAbridged.pdf.

IMHO, since the OP wants to apply linear algebra to real world problems, a better approach is to go with a matrix analysis book. Strang is very popular, but my favorite is http://matrixanalysis.com/Contents.html. Axler is a few notches higher in terms of abstraction. Hence, you won't learn lots of important practical results about matrices. In case of going with Axler, I'd use the previous edition. It's a shame they have ruined the typesetting by adding so many distracting color boxes and different fonts.

Personally, I'd go with Hubbard & Hubbard: https://matrixeditions.com/5thUnifiedApproach.html. It's a work of art that takes you from pre-calculus till multivariate calculus and analysis, along with all necessary linear algebra. Great mix of rigor, intuitions and practical details. At this level, as Hubbard points out, it's very useful to combine linear algebra with calculus & real analysis.

Another vote here for the matrixanalysis.com book. It is a really excellent book and takes a reasonably pragmatic approach to linear algebra. Coming from a programming background, you're more likely to find some things that resonate with you here. For example, this is one of the few introductory linear algebra books that deals with sensitivity analysis, which is useful to think about when dealing with floating point arithmetic instead of real arithmetic.

Would Hubbard and Hubbard be too difficult if I found Spivaks calculus too difficult?

I have been struggling to find a linear algebra book that isn't too abstract or too verbose. I did take LA and calculus a decade ago and I am trying to build up a background strong enough for probability and statistics.

It depends. Skim through H&H to see. I find it more intuitive and modern, but it also covers way more territory. At some point, the material will be hard because it's very advanced mathematics.

However, by then perhaps you have already adjusted. There's also a solution manual. Furthermore, many difficult proofs are in the appendix. So it's more of a calculus book if you want to ignore the analysis part.

Thanks for the reply I will definitely go check it out. I don't mind putting in the time and effort and a solution manual will definitely help.

No worries. Are you aiming at statistics? If so, what branch?

There might be other quicker ways to bootstrap. Then, you can come back to H&H.

I am hoping to get some experience with probalistic modelling, maximum likelihood etc. I work in bioinformatics and spent most of my time on algorithms development but probability/stats/ML are becoming the norm now. I find it hard to follow papers and develop new methods.

I work in that area :)

The skills needed will vary a lot. Hence my concern about studying H&H. It's a good idea, as real analysis is the foundation. But it will take too much of your time to get to something useful. Probably you should try to learn more applied material in parallel and let both threads merge in the future.

For maximum likelihood, you need to learn convex optimization right after real analysis. The canonical reference is [1], but there's also a very simple and pragmatic linear algebra textbook by the same author that also covers some of the optimization basics [2]. This might be a good entry point, certainly easier than Spivak or H&H. There's also [3,4], which you probably know about. These are great and emphasize the modeling part. Maximum likelihood (via EM) is in the appendix, and you don't really need to know a lot of math to get going.

If you prefer a Bayesian or a variational point of view, modeling is the really important part. MCMC and message passing algorithms tend to be reused. For high level modeling of study results (e.g. differential expression on complicated designs), Gelman's Stan books [5] are a delight to learn from. If you need to roll your own custom inference, you should learn about graphical data structures such as factor graphs [6,7]. Here, knowhow from H&H is also required.

[1] https://web.stanford.edu/~boyd/cvxbook/

[2] https://web.stanford.edu/~boyd/vmls/

[3] http://eddylab.org/cupbook.html

[4] https://www.cambridge.org/core/books/problems-and-solutions-...

[5] http://www.stat.columbia.edu/~gelman/books/

[6] http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

[7] https://www.inference.org.uk/mackay/itila/book.html

Thanks so much, it's nice to get suggestions from someone in the same area!

These are the most helpful and practical suggestions I have encountered. You've hit the nail on the head with the exact problem I have been having working through books like Spivak and Axler. It always felt like I wasn't learning anything practical towards my work and that anything useful was a long ways away. I do enjoy the books and the material and the suggestion to pursue them in parallel is something I wish I thought about.

I will definitely check out all of those links.

H&H is an absolute gem

I don't believe it's open access, or at least I see no download link on that page.

I don't see the link either, but I also do recall him releasing it for free.

You can find it on the Internet Archive:

https://archive.org/details/SheldonAxlerAuth.LinearAlgebraDo...

Determinants are usually introduced in Linear algebra out of the blue because you can't get to other important topics in Linear Algebra without them. Calculating them is a complex mess best left for a calculator. Sheldon teaches Linear Algebra as a theoretical math course, along the lines of learning Abstract Algebra. He approaches those other important topics from a different direction entirely, and determinants are just a trivial part of his book because of the different approach.

Reply on news.ycombinator.com