Comment by nextos | Hacker News Reader

Comment by nextos | Hacker News Reader

Parent Full thread

The philosophy of LADR is described in his paper Down with determinants: https://www.axler.net/DwD.html. In short, he thinks they obscure proofs. I love the book, but AFAIK only the compactified version (excluding all proofs, examples, and exercises, along with most comments) is open access: https://linear.axler.net/LinearAbridged.pdf.

IMHO, since the OP wants to apply linear algebra to real world problems, a better approach is to go with a matrix analysis book. Strang is very popular, but my favorite is http://matrixanalysis.com/Contents.html. Axler is a few notches higher in terms of abstraction. Hence, you won't learn lots of important practical results about matrices. In case of going with Axler, I'd use the previous edition. It's a shame they have ruined the typesetting by adding so many distracting color boxes and different fonts.

Personally, I'd go with Hubbard & Hubbard: https://matrixeditions.com/5thUnifiedApproach.html. It's a work of art that takes you from pre-calculus till multivariate calculus and analysis, along with all necessary linear algebra. Great mix of rigor, intuitions and practical details. At this level, as Hubbard points out, it's very useful to combine linear algebra with calculus & real analysis.

Another vote here for the matrixanalysis.com book. It is a really excellent book and takes a reasonably pragmatic approach to linear algebra. Coming from a programming background, you're more likely to find some things that resonate with you here. For example, this is one of the few introductory linear algebra books that deals with sensitivity analysis, which is useful to think about when dealing with floating point arithmetic instead of real arithmetic.

Would Hubbard and Hubbard be too difficult if I found Spivaks calculus too difficult?

I have been struggling to find a linear algebra book that isn't too abstract or too verbose. I did take LA and calculus a decade ago and I am trying to build up a background strong enough for probability and statistics.

It depends. Skim through H&H to see. I find it more intuitive and modern, but it also covers way more territory. At some point, the material will be hard because it's very advanced mathematics.

However, by then perhaps you have already adjusted. There's also a solution manual. Furthermore, many difficult proofs are in the appendix. So it's more of a calculus book if you want to ignore the analysis part.

Thanks for the reply I will definitely go check it out. I don't mind putting in the time and effort and a solution manual will definitely help.

No worries. Are you aiming at statistics? If so, what branch?

There might be other quicker ways to bootstrap. Then, you can come back to H&H.

I am hoping to get some experience with probalistic modelling, maximum likelihood etc. I work in bioinformatics and spent most of my time on algorithms development but probability/stats/ML are becoming the norm now. I find it hard to follow papers and develop new methods.

I work in that area :)

The skills needed will vary a lot. Hence my concern about studying H&H. It's a good idea, as real analysis is the foundation. But it will take too much of your time to get to something useful. Probably you should try to learn more applied material in parallel and let both threads merge in the future.

For maximum likelihood, you need to learn convex optimization right after real analysis. The canonical reference is [1], but there's also a very simple and pragmatic linear algebra textbook by the same author that also covers some of the optimization basics [2]. This might be a good entry point, certainly easier than Spivak or H&H. There's also [3,4], which you probably know about. These are great and emphasize the modeling part. Maximum likelihood (via EM) is in the appendix, and you don't really need to know a lot of math to get going.

If you prefer a Bayesian or a variational point of view, modeling is the really important part. MCMC and message passing algorithms tend to be reused. For high level modeling of study results (e.g. differential expression on complicated designs), Gelman's Stan books [5] are a delight to learn from. If you need to roll your own custom inference, you should learn about graphical data structures such as factor graphs [6,7]. Here, knowhow from H&H is also required.

[1] https://web.stanford.edu/~boyd/cvxbook/

[2] https://web.stanford.edu/~boyd/vmls/

[3] http://eddylab.org/cupbook.html

[4] https://www.cambridge.org/core/books/problems-and-solutions-...

[5] http://www.stat.columbia.edu/~gelman/books/

[6] http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...

[7] https://www.inference.org.uk/mackay/itila/book.html

Thanks so much, it's nice to get suggestions from someone in the same area!

These are the most helpful and practical suggestions I have encountered. You've hit the nail on the head with the exact problem I have been having working through books like Spivak and Axler. It always felt like I wasn't learning anything practical towards my work and that anything useful was a long ways away. I do enjoy the books and the material and the suggestion to pursue them in parallel is something I wish I thought about.

I will definitely check out all of those links.

H&H is an absolute gem

Reply on news.ycombinator.com