Linear algebra done right | Hacker News Reader

Linear algebra done right

linear.axler.net

281 pointsthe-mitr116 comments

Open articleView on HN

Last night I was looking into what to read after or along with 3Blue1Brown's series of Linear algebra videos [1]

The contenders seems to be:

- Linear Algebra Done Right - Sheldon Axler

- Liner Algebra Done Wrong - Sergei Treil

- Introduction to Linea Algebra - Gilbert Strang

- Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd and Lieven Vandenberghe

[1] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

+1 for Strang.

Strang is simpler and clearer. Axler is more advanced in the sense that it doesn’t tie it to matrices. Strange is a “first course” book, Axler is a second course.

Axler does limit itself to vector spaces over real and complex fields, though.

That‘s fine, but I would have appreciated notices, which proofs and theorems do not hold in the general case.

It‘s an exercise for the reader.

This is basically a request for a graduate course, though. Definitely not a first course in linear algebra.

TBH, I don’t think there are surprises in the “general case” (whatever that is… modules over a PID??) that you can’t see by understanding the real&complex situation.

No, but a second course. Still firmly in undergraduate territory, second semester at my university. Most undergraduate linear algebra textbooks also contain this, unless they are matrix computation heavy (usually for the first course).

That’s not true at all. He does precisely the thing you want him to do and also gives a pedagogical justification in the preface.

  > This book usually develops linear algebra simultaneously for real and complex vector spaces by letting  F denote either the real or the complex numbers. If you and your students prefer to think of F as an arbitrary field, then see the comments at the end of Section 1A. I prefer avoiding arbitrary fields at this level because they introduce extra abstraction without leading to any new linear algebra

And the remarks at the end of 1A are that if you want to, you can think of F as an arbitrary field everywhere except the sections on inner product spaces and where the given field is C you can frequently also use any other algebraically closed field.

Depends how you think. I found Strang impenetrable and Axler simple and lucid. Some people seem to find abstract vector spaces weird and unmotivated without doing a load of stuff with lists and grids of numbers first. I find determinants weird and unmotivated without learning exterior algebra first. I wish Axler had been my first course.

Lorenzo Sadun's Linear Algebra: The Decoupling Principle would probably be enough too if you added something about determinants.

+1 for Boyd

You zoomers are making a list of linear algebra books and not citing Lang? Get off my lawn ;)

Would you mind sharing your thoughts more?

Wikipedia says "The book has a pure, proof-heavy focus and is aimed at upper-division undergraduates who have been exposed to linear algebra in a prior course." [1], so it seems to be a different category of book?

[1] https://en.wikipedia.org/wiki/Linear_Algebra_(book)

I really recommend Matrix Analysis and Applied Linear Algebra by Carl Meyer. It's both concise and comprehensive. Strange is very good, but feels kinda vague and long winded in comparison (very good for a high level understanding of the tools you're dealing with)

Is this the book you are referring to? https://epubs.siam.org/doi/book/10.1137/1.9781611977448. It has 991 pages, not exactly what I would call concise.

I recommend getting the 1st edition. I read the 1st edition and loved it. Here is a relevant review of the 2nd edition https://janert.me/blog/2025/book-review-matrix-analysis-and-...

Thank you so much for the review link!! No ordinary reader would expect a complete rewrite from the first to the second edition.

Oh wow! That review was incredibly useful. I had seen there was a second edition and was eager to buy it next I came to the US - but was hesitating on dropping so much money on it.

Looks like it may be a useful read after I've mastered the first edition more :)

The FFT and SVD chapters have actually ended up being the basis of a few research projects I've done. The writing really has a way of reframing concepts in very insightful ways.

The layout of the book is very spacious,not cramped, with plenty of room for your own notes.

If you liked 3B1B and prefer intuition/applications-heavy view, then definitely Strang over Axler. Check out especially his newer textbook "Linear Algebra and Learning from Data".

Axler is more of a pure math textbook - if you want to dive more into proofs and abstractions.

I would also say Axler is much better prep for higher level applied math, as well as pure. If you are interested in how the big ideas of linear algebra extend to things like Fourier analysis it's very helpful to see the more abstract explanation of vector spaces.

Not everybody is ready for that. Certainly not as a first course.

Strang's lecture series are a nice and friendly accompaniment, especially if you don't have a reading group https://www.youtube.com/watch?v=7UJ4CFRGd-U&list=PL221E2BBF1...

LADW and LADR are great too, for an honors approach with more focus on proofs. To me it would make more sense on a second pass.

I strong second Strang. His book is the best first introduction to linear algebra, with "Done Right" marketing itself as a second course. Axler is notoriously shy with matrices, but Axler introduces them up front and uses them for the rest of the book.

The nature of textbooks is that each one is better suited for a certain profile of reader. It depends a lot on the way the reader has learnt to learn things until that point in their life.

If you liked 3B1B's style, you will prefer strang over axler. Axler and treil to a greater extent focus on bringing out the abstract elegance and the kind of rigour a math major enjoys. Strang's book also has videos accompanying - on MIT OCW.

B&V VMLS on your list is interesting - they focus a lot on real-world instantiations of the concepts and have you code up things in the (excellent) exercises. Depending on your goals, you can do only this, or strang and then this. Definitely look at the exercises in any case though.

As a math educator, I strongly dislike both Strang and Axler for a 1st course. I've heard great things about Strang's lectures, but his book is disorganized and too heavy on computation. Axler's book is wonderful, but as explicitly stated on the back cover, it's designed for a 2nd course and primarily aimed at math majors.

I recommend and teach my YouTube Live series out of Fraleigh [1], but unfortunately it's out of print. Lay seems to be a good modern alternative.

[1] https://linear.mathcanbeahobby.com

Apparently, it's on Internet Archive: https://archive.org/details/linearalgebra0000fral/

Appears only the front-matter of the book is available on IA (limited preview); the content was omitted.

agree, tried both and they are hard as a starter.

It may simply be that linear algebra is a difficult topic to become introduced to, and even the best teaching tools cannot make it easy.

Hard disagree. Linear algebra is one of the easiest topics in mathematics. In my experience, 3Blue1Brown videos are a great simple introduction. Beyond that, Kostrikin’s books.

What do you think of the other Fraleigh books?

Strang's OCW lectures are fantastic. His voice rings "combinations of columns" in my head almost ten years later whenever I do a matrix or matrix-vector multiply on paper. I found the textbook was a good accompaniment for the lecture course, but maybe not a good book to use on its own for self-study. (IIRC, when following the lectures, you jump around the chapters a bit out of order [or, you read the chapters non-linearly]. So the disorganized thing may be true.)

I agree Strang is awful, the department I taught in used Fraleigh/Lay in the past. I used a book by A.O. Morris.

We used Lay as well, it's still behind me on the shelf there. I haven't done much comparison but it would be interesting to revisit these and see if I can make them stick better. Everything always feels familiar but I wouldn't really be able to use or explain many of the techniques given e.g. a blank piece of paper.

LADW saved me in undergrad, but I was pretty much exactly the target audience in an honors-level freshman math course:

"[per Treil, LADW is for] a student who, while not yet very familiar with abstract reasoning, is willing to study more rigorous mathematics than what is presented in a “cookbook style” calculus type course."

But yeah it's really attempting to introduce you to higher mathematics rather than get you comfortable doing linear algebra per se.

I've taught out of the first three of these. If I had to pick one for a first study, I'd vote for Strang -- and watch his videos while you go. Our second-semester mostly-math-majors linear algebra course uses Axler, which I think is nice for the purpose, but our students already have done a semester of computational stuff first. (Though the complete absence of any computations in the book means they don't always connect the material from the two courses very well.)

I found Linear Algebra by Friedberg, Insel and Spence to be excellent. Very clear, modern notation, great exercises. It's also what Tao lectured from in 115A: https://www.math.ucla.edu/~tao/resource/general/115a.3.02f/

+1 I loved FIS for my Advanced Linear Algebra class

FIS was the text for my first “real” mathematics class and it is near and dear to me.

I ended up getting a math PhD so it must have done somethings right!

Yes, it was absolutely great! It's a good mix of theory and computation.

Since it doesn't neglect computation, it's great as an introductory text.

After that, and some other math courses you can move on to Linear Algebra Done Right.

I enjoyed "No bullshit guide to linear algebra". And I wish "Coding The Matrix: Linear Algebra Through Computer Science Applications" still had the autograder working. Though I wonder if I could throw some agents at replacing it....

I am a very big fan of Axler's approach, because to me it really gets to the (abstract) essence of things. When I truly started to "get" this approach, it really made linear algebra an important part of how I think.

I first learned from self studying Finkbeiner’s “Introduction to Linear Transformations and Matrices” which I thought was very good. Great exercise's. Dover still prints it for pretty cheap. He does a cool thing introducing all sorts of theory of linear transformations, then later showing that matrices are the way to encode them once you chose basis for domain and codomain. I liked that a lot, felt like it removed any magic from matrices. Also a fan of Axler though.

You might enjoy my book https://pimbook.org/, which includes chapters on linear algebra.

Sorry to bother but is there hope we are going to see Practical Math for Programmers published any time soon?

"Linear Algebra and Its Applications" by David Lay is the best introductory linear algebra textbook, hands down. Love it.

I had this as my very first textbook in community college. Loved it!

I hear recommendations for this book all the time; I've cover-to-covered Strang several times and have digested maybe 20% of Axler (I just dip into random sections of LADR when something annoys me in a problem set or whatever), and I'm super curious why the aura on Lay's book is so positive.

(I don't doubt it at all! I'm just curious.)

Lay explains things very clearly and it's very good for going back to when I need to review something. Lay covers all the foundational linear algebra topics you would need for say, ML or graphics programming. I found that roughly the first half of LADR felt like a review of what Lay covered.

You won't get anything new from Lay's book if you've done Strang. It is a basic textbook and doesn't cover anything advanced. It's just a good textbook.

As someone who's watched all Strang's lectures and read most of Axler's book, I found Linear Algebra and Its Applications to give the most interesting treatment on the topic. What made it so special was its clear explanations and the use cases it introduced.

The applications not only provided the context to get a deeper understanding of the concepts, it also made the math relevant to the real world. By showing a range of problems, from the chemical stoichiometry to the algorithmic SVD, it inspires the reader to look for more problems that can be solved with the toolset being introduced.

For a second pass to linear algebra, “Linear Algebra via Exterior Products” from Winitzki is great

if self teaching you need a book that holds your hand and doesnt try to put you through the usual trials of "lets see if you can figure this out yourself.. oh you cant? maybe repeat the subject next term", or major proofs left as exercises for the reader. get a friendly hand-holdy book. im reading linear algebra, theory, intuition, code by mike cohen atm (up to ch4) and so far between that and a chatbot for clarification/links, im re-learning the L.A. i should have learn't at uni but for whatever reason just did not understand back then. those courses move fast and if you get stuck somewhere its game over. i think L.A. is way harder than calculus as a beginner, but once things start to click and fall into place it starts to feel easier.

> lets see if you can figure this out yourself..

If you are SELF-teaching, you need to learn how to figure things out.

When you can’t, there’s plenty of sources. It’s not the 1990s. Your favorite LLM probably gives good explanations of linear algebra ideas even on fast mode. Gemini was surprisingly good for me.

taking it upon yourself to self teach is figuring things out for yourself, no need to make it harder than it needs to be.

The thing about linear algebra is that it isn't really a cohesive subject in the same way that calc is. This is why you get as many perspectives as you get commenters.

There are some central concepts, and which ones matter to you depends on what you're planning to do with your future. The key points are:

* Matrices/vectors as grids of numbers/computational tools.

* Matrices/vectors as positions and transformations of those positions.

* Matrices/vectors as more abstract geometric objects

* Matrices/vectors as algebraic objects.

There is no 'done right' imo. My sense is LADW is probably the best option for a motivated honors math student, because unlike Axler the author doesn't hate determinants for whatever reason. Strang would work well for engineers. Axler is mostly concerned with the last two, but IMO this makes him kinda niche.

honestly students should just study ode's to motivate and learn linear algebra which otherwise is lacking substance, and peek into pde's to see why vector space necessary and insightful especially for nonlinear pde.

For math majors, LADR is far better than LADW, almost most of LA books use the determinant to prove the advanced stuff (diagonlaization etc), what sets LADR apart is the intentional delay of introducing determinants in order to prove (and hence understand) what's the goal behind concepts such as diagonalization, I love this method.

Axler is usually recommended as a great second book for linear algebra (unless you're extremely comfortable with proofs).

For a first exposure, these days it's hard to do better than Math Academy [0]. It'll diagnose all your weak points (including what prereqs you might not have) and handle all the scheduling using spaced repetition and knowledge about how the topics relate to each other.

All you have to do is keep showing up and do the work for 30+ minutes per day.

[0] https://www.mathacademy.com/courses/linear-algebra

Linear Algebra by Jim Hefferon can be read online for free.

Note that "done right" means done with Axler's completely subjective and unusual hatred of determinants, chronicled here [0]. It is in no way "done right" in some definitive, rigorous way; most math professors I have spoken to either strongly disagree with the presentation or have no particular preference.

[0] https://www.axler.net/DwD.html

Yeah I feel like this "done right" part is responsible for most of the popularity of this book. Makes the reader think they've been learning it wrong. Kind of like these clickbait videos "you've been folding your laundry wrong your whole life!" or whatever

Or "this one simple trick makes Big Linear Algebra hate you"

The popularity of Linear Algebra Done Right (IMO) comes from two sources:

1. Excellent exercises. Challenging. Really make you put the concepts together.

2. Good, opinionated pedagogy. If you agree with the philosophy (among other things, determinants are not a beginner tool), the explanations are good.

LADR is hardly the only book to eschew determinants for a long time. IIRC Lang takes a similar approach, but is not as digestible.

Determinants are easy to use but very hardly to grasp intuitively, this is not a minority point of view. See countless of StackOverflow questions begging for a conceptual exposition of determinants.

The easiest conceptual handle is geometric: volume expansion, but seeing how this is related to the combinatorial sum over all permutations, or how those two point of views are related to the algebraic one (that a set of equations having a solution or not), is not easy to see even in the 2D case.

I won't be surprised if math professors don't have this issue like you said (especially if someone is comfortable with wedge products), but the vast majority of newcomers who are interested in understanding why something works rather than just how to use it struggle all the time with determinants.

I think the volume explanation is one of the most intuitive pieces of math in existence, personally! Uninvertibility of a tranformation corresponds to a volume of zero because the transformation must squish two dimensions together, leaving them impossible to differentiate, det(AB) = det(A)det(B) because applying two transformations applies their scaling successively, det(A^-1) = 1/det(A) because you have to undo the scaling to invert a transformation etc. I don't think the permutation definition is even strictly necessary; if I recall correctly Linear Algebra Done Wrong defines the determinant in terms of its geometric definition and develops its formula from the properties it must have. I think that the concept is well worth the investment of initial confusion. Axler disagrees, however.

But if you see the geometric view then the permutation sum, for certain obsessive learners they want to see why they are equivalent and thats the hard part.

It seems to me more like determinants are often badly motivated. The way I remember it from Apostol's Calculus was like "a volume multiplier would be very useful; it needs to be a signed volume for linearity, which implies antisymmetry. Here are axioms collecting these requirements. They're uniquely satisfied by the determinant. Proof: ..."

Agreed that it should help if you got to learn wedge products first (I didn't).

This is indeed a good concise description of how to connect the two, but even making peace with this, there is something still magical in how the permutations in the sum cancel neatly (in an inclusion-exclusion kind of a way) to get the volume.

I found that “properties of the determinant uniquely determine this formula that I guessed” approach to determinants to be extremely unconvincing when I was learning linear algebra.

This might come down to details of how you explain it: iirc Apostol took those basic moves (axioms) and calculated what the formula would have to be, rather than starting with a formula and checking that it has the properties of a signed volume.

But I don't know, it's been a very long time for me. It's good to have a variety of approaches to the subject.

Yes, but that doesn't prove that the determinant is actually the real signed volume multiplier.

I think this is what led me to feel unhappy about determinants when I was a first-year university student. You need to actually prove that the determinant is the volume of the N-dimensional parallelepiped, and the axiomatic proof doesn't do that.

So you need basically two extra lines after proving those things so that people can say "okay, the determinant eats ignores all input vector non-orthogonality so that it gives volume".

Maybe you need to speak to better math professors.

Previously on Hacker News:

Linear Algebra Done Right 58 points, July 2023, 4 comments https://news.ycombinator.com/item?id=36576114

Linear Algebra Done Right – 4th Edition, 631 points, Oct 2023, 294 comments https://news.ycombinator.com/item?id=38060159

Linear Algebra Done Right [pdf], 85 points, Sept 2024, 39 comments https://news.ycombinator.com/item?id=41416799

Previously in my ~/Downloads:

linear_algebra_done_right.pdf, 0 pages read, July 2023

linear_algebra_done_right (1).pdf, 0 pages read, Oct 2023

linear_algebra_done_right (2).pdf, 0 pages read, Sept 2024

Downloading (3) now.

This reply made me die of laughter lol

Nice to see a fellow ebook hoarder here. I always get that magpie impulse to download any book that looks or sounds interesting, even though I may never read them.

This is supposedly based on Sheldon Axler's earlier and shorter paper "Down With Determinants!" [0]. I lectured mathematics for a while at a "former polytechnic" and used to enjoy leaving print-outs of this sort of paper in the faculty communal areas.

[0] https://www.axler.net/DwD.html

For those who find Linear Algebra Done Right too much to start with, and those who don't get why Strang starts with matrices, I can't recommend more "The dark art of linear algebra" read this first. With this you can then tackle every other book on the topic more easily

My personal favorite is No Bullshit Guide to Linear Algebra - it gives a really good overview of math fundamentals and overall strikes a good balance between keeping things simple and giving enough insight to comfortably dig deeper in the topic.

If anyone is interested, check out the PDF preview here https://minireference.com/static/excerpts/noBSLA_v2_preview....

and the printable concept maps here: https://minireference.com/static/conceptmaps/linear_algebra_...

Nice seeing you on HN. I thoroughly enjoyed the No Bullshit LA and Maths & Physics books. I wish more books would include concept maps, I find them a very useful tool for understanding how things fit in the broader picture.

Yes, concept maps are the bee's knees.

I remember from my tutoring days how useful they were to organize the different concepts covered in each lesson: I would start with a blank sheet and make the student add concepts to it as the lesson progressed, then by the end of the lesson use the concept map to review what we learned. Specifically, I would ask them to explain in their own words each "arrow" which was a great way to uncover misconceptions and solidify the material.

Once I'm done with editing the current book[1], I hope to have time to work on making dynamic concept maps that you can click on and explore/zoom-in on. I feel it would be cool to jump between detailed view (concepts), intermediate scale (topics), and high-level view (subjects).

[1] https://noBSstats.com

When it's described in plain English [0], with lots and lots of examples, that's when I call it done right.

This is the best I know: https://www.youtube.com/watch?v=Fnfh8jNqBlg&list=PLlXfTHzgMR....

[0] I meant natural language

Absolutely love Dr. Grinfeld! I watched some of his differential geometry series and his explanations are very accessible!

Overrated and tendentious book. There are many better linear algebra texts. His polemic against determinants is poorly motivated, misguided, and distracting. The writing is quite formal and not terribly inspiring. The coverage is adequate but nothing more.

> His polemic against determinants is poorly motivated, misguided, and distracting.

What polemic? Defining the determinant as the unique multilinear alternating form satisfying certain properties is very normal (and in fact the only way that really makes sense for both finite- and infinite-dimensional vector spaces). There are zero unusual things with this book imo.

This polemic: Sheldon Axler -- Down With Determinants

https://www.axler.net/DwD.html

A strange and unpopular opinion.

Something else you recommend?

In a computer science context, I have to plug the old FLAME group for publishing and teaching the topic so well. - digging into parallelizing computations efficiently before neural nets took off around the 2014 time.

LibFlame has long been abandoned now but their courses were very strong when you had access to the professors. They have been rebranded as the Science of High Performance Computing (SHPC) group.

https://github.com/flame/libflame

https://shpc.oden.utexas.edu/

Lately been deep diving into linear algebra. And a way which i engage with it is that I tell AI to generate interactive examples + questions on Lean or Haskell. Its so fun, just deriving the intuition in these languages.

Check out page 196 for a Shakespearean style sonnet on the Cauchy-Schwartz inequality, courtesy of Chat-GPT.

These days, linear algebra done right should be accompanied with some CAS to view how algorithms are used.

Possibly paired with some numerical algebra free text (many on the Internet)

If you want to dive deeper into applications, refer to Linear Algebra and Geometry by I. Kostrikin, Yu I Manin.

This advanced textbook on linear algebra and geometry covers a wide range of classical and modern topics. Differing from existing textbooks in approach, the work illustrates the many-sided applications and connections of linear algebra with functional analysis, quantum mechanics and algebraic and differential geometry. The subjects covered in some detail include normed linear spaces, functions of linear operators, the basic structures of quantum mechanics and an introduction to linear programming. Also discussed are Kahler's metic, the theory of Hilbert polynomials, and projective and affine geometries. Unusual in its extensive use of applications in physics to clarify each topic, this comprehensice volume should be of particular interest to advanced undergraduates and graduates in mathematics and physics, and to lecturers in linear and multilinear algebra, linear programming and quantum mechanics.

My alma mater took an interesting approach in that they dived right into vector spaces right away. At the time we used Elementary Linear Algebra by Nicholson in the prof's custom order, but it seems they've since written an open textbook based on that approach:

https://ruor.uottawa.ca/items/f66a4ede-e276-486c-9067-9621d5...

Really felt like being thrown in the deep end at the time, especially since it was my first exposure to proofs. Very rewarding when it finally started coming together, though.

I passed the class just because of how good the book is.

I got halfway though the exercises with the help of a reading group. They were very hard, bit thought provoking, so I would definitely recommend. Don't feel discouraged if you get stuck and try not to look at the solutions right away.

In the 1980s, being not entirely adept at mathematics I recall scouring every library and bookstore I could find for any snippet that would explain a proof, or even a concept, so that I could understand it. Videotaped lectures by other professors were sometimes available on campus too.

On a daytime episode of David Letterman, Isaac Asimov predicted fiber optics would one day bring about television studios in people's homes: https://youtu.be/cIB1b_8hqB0?si=212sGzZ71VIZORML&t=696

All sources of understanding are so very much appreciated.

I found it really insightful (and always overlooked) to distinguish between vector and co-vector spaces. It doesn't necessarily produce new knowledge, but makes things more clear.

Yeah - otherwise it’s bizarre to have row vectors and column vectors.

To me it doesn't seem wrong to call covectors "row vectors", but one shouldn't simply convert them to each other with no good reason.

I wanted to learn the underlying principles of LLM/AI and got myself Shilov's book. Wow that was so thick, each paragraph took a while to figure. This could be a nice option..

Thanks!

Shilov is a very popular recommendation, because it‘s good and very, very cheap (a Dover).

This is a holy book for a lot of game developers.

Really? Why ? It's heavy on theory and proofs.

I am assuming game developers are interested in the applications of Linear Algebra and there are many other books that are far suited for that.

Anyone from game dev community care to elaborate ?

Essential Mathematics for Games and Interactive Applications, and Mathematics for 3D Game Programming and Computer Graphics.

Interesting! The one with Gilbert Strang was good for me...

"Human verification failed" - so this is a broken link.

What do folks think of Hefferon's _Linear Algebra_?

It's a great introduction, and better yet, it's completely free!

you only need like 5 simple concepts to understand linear algebra in ai/ml. you can learn it a week using chatgpt.

i think lot of ppl are under the impression that long courses are a prerequiste to even start exploring ai/ml.

doing long prerequeste courses is why lot of ppl drop out even before getting to ai

Kindle format link == 404

i literally threw this book in the trash cause it was too dense and pretentious.

Do not get the latest edition, the layout and typesetting is atrocious!

My bag of tricks is better than your bag of tricks. Alright.

As with most textbooks, it fails to motivate why reading it is worth the investment. Perhaps it is a millennial old tradition of the Greek mystery schools, that the rite of passage came by proving your commitment to material knowledge without anything but fate in the school itself as motivation.

Rigor before Worth.

(Yes this is a pet peeve of mine :)

Reply on news.ycombinator.com