Hacker News Reader

Comment by isaacimagine | Hacker News Reader

I enjoyed Linear Algebra Done Wrong [0], to be used in combination with a more traditional textbook, like Linear Algebra and its Applications [1] (which has some good diagrams). I've already seen it mentioned, but I'd like to add that 3b1b's Essence of Linear Algebra [2] videos are well made and make for a good supplementary resource early on.

[0]: https://www.math.brown.edu/streil/papers/LADW/LADW_2017-09-0...

[1]: https://www.amazon.com/Linear-Algebra-Its-Applications-5th/d... — PDFs exist.

[2]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

View on HN

The title of Linear Algebra Done Wrong is an unacknowledged nod to Sheldon Axler's Linear Algebra Done Right. I know Sheldon; he believes it's a crime to teach people determinants. I teach people determinants. Wrong features determinants in Chapter 3.

I was once part of an interactive learning software demo, where Sheldon had provided the sample linear algebra problem. I solved it in seconds using determinants. That really made my day.

>I know Sheldon; he believes it's a crime to teach people determinants.

Any way to explain to a lay person why?

The book is open access (https://linear.axler.net/), I am going to quote directly the author in the preface:

<< all linear algebra books use determinants to prove that every linear operator on a finite-dimensional complex vector space has an eigenvalue. Determinants are difficult, nonintuitive, and often defined without motivation. To prove the theorem about existence of eigenvalues on complex vector spaces, most books must define determinants, prove that a linear map is not invertible if and only if its determinant equals 0, and then define the characteristic polynomial. This tortuous (torturous?) path gives students little feeling for why eigenvalues exist. In contrast, the simple determinant-free proofs presented here (for example, see 5.21) offer more insight. Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra— understanding the structure of linear operators.>>

If you like mathematics, it is actually a pretty nice book.

I don't believe it's open access, or at least I see no download link on that page.

I don't see the link either, but I also do recall him releasing it for free.

You can find it on the Internet Archive:

https://archive.org/details/SheldonAxlerAuth.LinearAlgebraDo...

The philosophy of LADR is described in his paper Down with determinants: https://www.axler.net/DwD.html. In short, he thinks they obscure proofs. I love the book, but AFAIK only the compactified version (excluding all proofs, examples, and exercises, along with most comments) is open access: https://linear.axler.net/LinearAbridged.pdf.

IMHO, since the OP wants to apply linear algebra to real world problems, a better approach is to go with a matrix analysis book. Strang is very popular, but my favorite is http://matrixanalysis.com/Contents.html. Axler is a few notches higher in terms of abstraction. Hence, you won't learn lots of important practical results about matrices. In case of going with Axler, I'd use the previous edition. It's a shame they have ruined the typesetting by adding so many distracting color boxes and different fonts.

Personally, I'd go with Hubbard & Hubbard: https://matrixeditions.com/5thUnifiedApproach.html. It's a work of art that takes you from pre-calculus till multivariate calculus and analysis, along with all necessary linear algebra. Great mix of rigor, intuitions and practical details. At this level, as Hubbard points out, it's very useful to combine linear algebra with calculus & real analysis.

Would Hubbard and Hubbard be too difficult if I found Spivaks calculus too difficult?

I have been struggling to find a linear algebra book that isn't too abstract or too verbose. I did take LA and calculus a decade ago and I am trying to build up a background strong enough for probability and statistics.

It depends. Skim through H&H to see. I find it more intuitive and modern, but it also covers way more territory. At some point, the material will be hard because it's very advanced mathematics.

However, by then perhaps you have already adjusted. There's also a solution manual. Furthermore, many difficult proofs are in the appendix. So it's more of a calculus book if you want to ignore the analysis part.

Thanks for the reply I will definitely go check it out. I don't mind putting in the time and effort and a solution manual will definitely help.

No worries. Are you aiming at statistics? If so, what branch?

There might be other quicker ways to bootstrap. Then, you can come back to H&H.

I am hoping to get some experience with probalistic modelling, maximum likelihood etc. I work in bioinformatics and spent most of my time on algorithms development but probability/stats/ML are becoming the norm now. I find it hard to follow papers and develop new methods.

I work in that area :)

The skills needed will vary a lot. Hence my concern about studying H&H. It's a good idea, as real analysis is the foundation. But it will take too much of your time to get to something useful. Probably you should try to learn more applied material in parallel and let both threads merge in the future.

For maximum likelihood, you need to learn convex optimization right after real analysis. The canonical reference is [1], but there's also a very simple and pragmatic linear algebra textbook by the same author that also covers some of the optimization basics [2]. This might be a good entry point, certainly easier than Spivak or H&H. There's also [3,4], which you probably know about. These are great and emphasize the modeling part. Maximum likelihood (via EM) is in the appendix, and you don't really need to know a lot of math to get going.

If you prefer a Bayesian or a variational point of view, modeling is the really important part. MCMC and message passing algorithms tend to be reused. For high level modeling of study results (e.g. differential expression on complicated designs), Gelman's Stan books [5] are a delight to learn from. If you need to roll your own custom inference, you should learn about graphical data structures such as factor graphs [6,7]. Here, knowhow from H&H is also required.

[1] https://web.stanford.edu/~boyd/cvxbook/

[2] https://web.stanford.edu/~boyd/vmls/

[3] http://eddylab.org/cupbook.html

[4] https://www.cambridge.org/core/books/problems-and-solutions-...

[5] http://www.stat.columbia.edu/~gelman/books/

[6] http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

[7] https://www.inference.org.uk/mackay/itila/book.html

Thanks so much, it's nice to get suggestions from someone in the same area!

These are the most helpful and practical suggestions I have encountered. You've hit the nail on the head with the exact problem I have been having working through books like Spivak and Axler. It always felt like I wasn't learning anything practical towards my work and that anything useful was a long ways away. I do enjoy the books and the material and the suggestion to pursue them in parallel is something I wish I thought about.

I will definitely check out all of those links.

Another vote here for the matrixanalysis.com book. It is a really excellent book and takes a reasonably pragmatic approach to linear algebra. Coming from a programming background, you're more likely to find some things that resonate with you here. For example, this is one of the few introductory linear algebra books that deals with sensitivity analysis, which is useful to think about when dealing with floating point arithmetic instead of real arithmetic.

H&H is an absolute gem

Determinants are usually introduced in Linear algebra out of the blue because you can't get to other important topics in Linear Algebra without them. Calculating them is a complex mess best left for a calculator. Sheldon teaches Linear Algebra as a theoretical math course, along the lines of learning Abstract Algebra. He approaches those other important topics from a different direction entirely, and determinants are just a trivial part of his book because of the different approach.

What's the point of learning linear algebra without determinants? How else are you supposed to learn about eigenvectors?

An eigenvector of a linear operator is a nonzero vectors s.t. the operator multiplies it by a scalar. It's existence for a complex finite dimensional operator can be proven without the determinant.

The minimal and the characteristic polynomials can be defined and described without the determinant.

Where do you think you need the determinant?

Well at some point you may want to actually calculate the eigenvalues. Unless you just dive in and try and compute the characteristic polynomial directly.

I don't think Sheldon Axler wants to literally banish determinants, they are a useful tool.

But they do obscure the meaning in a lot of proofs. If you start using them immediately, it is also hard to motivate where they come from, what's the intuition behind them and how the students could arrive at the definition themselves.

And even if you use them and they produce short one-line proofs, you should also prove all the properties behind them first, which is where all the complexity is hiding.

The nice point about Linear Algebra Done Right is that it explains how Linear Algebra works without resorting to "determinant magic". In the end, you will also understand why determinants work so well in so many proofs.

In any case, his book is really suited for a second course in Linear Algebra, so maybe it is beneficial to start using determinants right away. Personally I understood them better with his book.

Disclaimer: I don't know Sheldon Axler, but I read his book.

this is definitely one of the problems I have always had. There seems to be so much "lore" hidden in various axioms about these things that often some derived concept like determinants that seems to relate to nothing intuitive are taught to the exclusion of insight generating concepts. They seem to think it's a clever way to teach to do this and then later on reveal the magic behind the curtains, saying "see this is how determinants work!". But for me I was lost and gave up trying to understand 6 lectures ago.

Agreed which is why I love Artin’s Algebra. It introduces the matrix determinant early and uses it as the prototypical example of a group homomorphism and the book uses matrix groups liberally.

Obviously not a beginner friendly book if you just want to understand matrix equations s. But I found the pedagogical approach great and there are some real gems in the exercises.

I found it a better for learning versus the standard recommendations of Axler/Lang/Halmos for LA. (Lang’s exposition is kinda weird and Halmos’s choice of exercises isn’t my favorite).

3b1b explains the meaning of a determinant so beautifully in his video.

Take a look at the comments on his YouTube video - there are tons of people that say this is the first time ever they came across such an explanation.

To echo sibling comment but make it more prominent - "Linear Algebra Done Right" is a great second book, I wouldn't use it as a first source of learning Linear Algebra. (I'm pretty sure it says as much in the intro, IIRC.)

I personally learned from Strang, then this book, which complemented each other very well. Strang is very focused on practicality, actually computing things, always seeing the matrix behind everything. "Done Right" is great at ignoring most of the "matrix view" of things completely in favor of pure linear transformations instead. Both views have their use - so learning one, then the other, is a great idea.

That is really a wonderful book which I perused when learning linear algebra, maybe a bit on the mathy side for OP expecially as he is asking a book for a "visual learner". Fortunately linear algebra can be grasped intuitively in dimensions above 3 even if it can't be visualized, but maybe I'm biased as it is bread and butter for me now.

Reply on news.ycombinator.com