The best linear algebra books | Hacker News Reader

The best linear algebra books

begriffs.com

108 pointsbegriffs40 comments

There's a introductory book that I would add to the "generalist" list, that is 'Linear Algebra Done Wrong' by Sergei Treil. It can be found freely here: https://www.math.brown.edu/~treil/papers/LADW/LADW.html

I've used it to prepare my linear algebra exam in an italian university (which usually emphasize theory over practice, even in engineering courses), and I've loved it. There are few exercises but they're to the point. Complex inner product, self-adjoint operators etc. are treated. I find the notation used in there particularly easy to understand, and proofs are elegant but very easy to follow.

I was on the fence about that one and ultimately took it off the list. It seems to be fairly numerical, sort of in defiance to Axler's "Done Right" book. But then as a numerical book how does it compare with the others in that category, or with the old Russian books? I ask quite honestly because I haven't gone through Treil's book. Does it hold its own?

Actually, I think it's very similar to Axler's in spirit. Not numerical at all (there is a description of the min squares problem, but that's it as far as I remember). Matrices are used more explicitely than in Axler, but only when necessary, because the author claims he's an operator theorist so the focus is on linear operators. Determinants are introduced as multi-linear functionals and are not avoided as in Axler. Supporting the fact that the book is quite theoretical, this is what you can find in the last chapters:

Chapter 6. Structure of operators in inner product spaces. Chapter 7. Bilinear and quadratic forms Chapter 8. Dual spaces and tensors Chapter 9. Advanced spectral theory

Unfortunately I'm not familiar with the Russian books.

I have skimmed some books about Linear Algebra, both in Italian and in English, and my favourite so far is "Geometria e Algebra Lineare" by Enrico Schlesinger.

The book is very well written but sadly it won't be enough to pass a typical italian Linear Algebra exam. This is because, in my opinion, examples are too vague or too "abstract": when you're first learning such subject, it would be better to have layman terms, that is simple examples with simple numbers. More complex examples might be okay too, but simple examples should come first.

Reference: http://www.zanichelli.it/ricerca/prodotti/algebra-lineare-e-...

Marco Abate's "Geometria" was also quite entertaining.

My favourite linear algebra book in italian is "Geometria e Algebra Lineare" by Abbena, Finno and Giannella. Really great book, with a wealth of exercises and extensive material.

I can also suggest the youtube lectures[0] by Francesco Bottacin, who actually was the one that eventually did my oral examination.

He's also written two books based on the course, the one with exercises[1] is very solid and I think you can pass the written exam by just doing it whole. They told me he is a wonderful lecturer. Pity that all this material is in italian.

[0] https://www.youtube.com/channel/UCpJGeVBfmf-6S3neeImAw1w/fee...

[1] https://www.amazon.it/Esercizi-algebra-geometria-Francesco-B...

You'd probably like Strang. He's an mit professor who shared his linear algebra course lectures. They're excellent.

http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

A one-edition back version of his book _Introduction to Linear Algebra_ is available for under $30.

https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S...

Unfortunately that book was too lightweight on theory to be of use to me, but it's certainly a very good book. I've recently started Strang's more advanced book on computational linear algebra/numerical analysis and it's pretty good though.

I actually don't like the Strang book and his MIT course.

I watched 6 or 7 of his lessons, and to be honest, I think they're nice if you already know the subject.

I have skimmed both an italian version of his book and an english one, still didn't like it.

The nice thing about the Schlesinger book is that it builds all of the concepts basically from scratch. Euclid’s Elements are quoted too, in the introduction.

Too bad it has poor examples.

That was not my experience -- I was utterly confused by my college linear algebra class which was taught as a bag of techniques. I couldn't figure out why anyone would care or what you would do with any of it. And I definitely didn't get the connections to the geometry that Strang goes over in Ch4 iirc.

Strang taught me linear algebra. His course -- and I read a couple other books afterwards -- is the only reason I understand it. Not only the how of the techniques, but why and what they do.

I don't suppose there's an english translation of the Schlesinger book? Google says no.

Linear Algebra Done Wrong is quoted at many places with another book - Linear Algebra Done Right. Did you study other book - any thoughts?

Given the frequency and popularity of these "math texts for programmer audience" threads on HN, I think there's a solid business model where programmers pay a onetime subscription fee for unlimited email/hangouts access to a math professor. The problem with learning math all by yourself is, to quote my favorite math professor- "math is a foreign language". So its a bit like trying to learn chinese sitting in a non-chinese setting, with only a chinese grammar text. You are simply not going to get very far. Far easier to buy a plane ticket to China and surround yourself with Chinese speakers, so you learn through osmosis - which is what the university experience is all about. But not everyone can take time off and go to a university. So then, atleast have a Chinese mentor on call, who can guide you through the difficult parts, tell you why things are the way they are etc.

I'd be glad to pay $1000 for a 1year unlimited email access to a math prof who can help me with some of the harder portions of measure theory and Banach spaces. Sign up enough students like myself and you have a lucrative gig.

As a former math professor, I'm not confident that this would be as scalable as you might be assuming. Given a nominal fee of $1k/year and a reasonable target of 100 such subscriptions, that is 100 people requesting "unlimited" amounts of your time in 1on1's. And decent 1on1's require non-trivial prep. And, given 100 customers, at least one is bound to push the limits of the "unlimited" premise.

The "unlimited" is mostly branding, like gmail's unlimited email storage. In practice, professionals like myself simply don't have the time to bug you at all hours of the day. I have a dayjob and maybe an hour I could devote to math every weekday. So worst case you'd receive 1 question a day...usually far less. Moreover, students tend to ask the same questions, so after the initial phase you can simply redirect most queries to preexisting URL. Also, you as a professor don't have to chime in for everything - have a graduate student preprocess the majority of the questions and you attack the harder 10%. Its definitely feasible. Chegg does something similar - what's missing is the quality and rankings

Or you could just take a class at a local university or community college.

You could contact the professor and see if they're okay with you auditing and using office hours and the like without paying or you could contact the university and see if you can pay to take a class without enrolling in a degree program.

So has the author read all these books? Is there a review? What makes these the best? Isn’t a significant part of the content in these books overlapping?

What is someone who isn’t a graduate student algebraist supposed to take away from a list of 33 books? At typical textbook prices, that’s like $4000 and 5 years of doing nothing but reading about linear algebra.

(Sorry if this sounds negative; I’m sure there’s some audience who will find this list useful.)

Honestly, I'm not even sure how much a review would help either. I struggled quite a bit with linear algebra in school with Curtis' Abstract Linear Algebra, and eventually I decided to hole up in the math library for the better part of a week and went through every linear algebra book they had (definitely more than 33) to find the best supplementary material. I guess it was a pretty hard class because about half of the other students had the same idea. None of us settled on the same book as our favorite.

My sample size is exceptionally small, but the opinion I took away was that linear algebra was just one of those things you have to plod through lots of books to find the one that works for you.

Valid question - a matrix of [ linear-algebra-topic x books ] would've been intuitive to know where it overlapped with other and a missing part.

I can't believe he didn't mention Gilbert Strang. This should be the starting point for everyone, IMO.

The oddball Russian books look very interesting, but for a computer guy to learn, I'd suggest Strang, then Trefethen & Bau (my favorite). If there is anything else worth knowing about numeric linear algebra it is in Gollub, or Demmel's applied numerical linear algebra (another unforgivable omission; he only helped write LAPACK!).

The problem books can also be important; they're all pretty good if you pick the problems right.

Not sure the highly theoretical stuff is actually worth anything as far as getting things done, unless you go on to study Algebra and mathematics in general. Ultimately the "real world" stuff where linear algebra is studied as systems of equations and geometric constructs is going to be closest to what you actually do with linear algebra on a computer.

> I can't believe he didn't mention Gilbert Strang. This should be the starting point for everyone, IMO.

Not everybody loves this book, you know. A lot of people find its sloppy, disorganized, chatty style quite irritating.

> Not sure the highly theoretical stuff is actually worth anything as far as getting things done, unless you go on to study Algebra and mathematics in general.

You've answered your own question. It's "highly theoretical", not "highly practical". The focus is different by definition.

Really appreciate this effort. My mathematical background is somewhat lacking and I'd love to pursue ML/Data science where I'm led to believe this is a prerequisite. Anyone able to tell me what other areas of math I might need as a baseline before diving in here?

Practically, a main difficulty in ML/Data science, is getting the data. Applying algorithms is largely limited to applying implementations of algorithms.

So, yes, it helps to be familiar with this level of stuff. But don't let it stop you tinkering with what you have access to.

If you have already studied linear algebra at an undergraduate level, read Trefethen and Bau’s book. Otherwise, perhaps find an undergraduate course somewhere first (OCW? Some MOOC?). Most of the stuff on this list is very abstract and technical, and won’t necessarily be relevant to you unless you want to become a professional mathematician.

My advice to you would be to start backwards. Find a few interesting-looking papers in the field and get a feel for the math involved and learn it. If you learn your mathematics as a tool to make sense of the papers you're reading, you're likely to be going down the right path for your goals.

That said, linear algebra and statistics (+ prereqs) are an absolute necessity.

Why no Halmos, "Finite-dimensional vector spaces"? It's pretty theoretical but very approachable also.

I own that book and have worked through a few chapters. It is one of the first linear algebra books in a "modern" style, being published in 1958. The reason it's not on the list is I just didn't see what it had to offer that isn't covered in later books. Happy to be convinced otherwise though.

I do have Halmos' linear algebra problem book on the list because it's quite good.

> later books

bourbaki is fairly early. the date on your site is related to translation.

although the halmos book appears wordy, i think he has a sound pedagogical style appropriate to his target audience, having glanced at the text mentioned here and really enjoyed his book on set theory.

> by choosing one road I am turning my back on a thousand others

ah, no need to paint such a glum picture: there are only a few facts that will appear and reappear in many of the theoretical books, b/c there is only one thing called, "vector space," one thing called, "hilbert space," and so on.

for hacking, of course, numerical methods are what many readers will be interested in...

> Weird Russian

this might be the best section, except for the categorical "weird" label. the iron curtain must have been sort of like a semi-porous membrane. probably just about every book list on math topics considered during the mid-20th-century ought to include some books by russian authors.

> The reason it's not on the list is I just didn't see what it had to offer that isn't covered in later books. Happy to be convinced otherwise though.

Halmos wrote Finite Dimensional Vector Spaces in 1942 -- shortly after he got his Ph.D. from J. Doob at University of Illinois -- at the Institute for Advanced Study in Princeton as an assistant to von Neumann.

The book is a mostly finite dimensional introduction to von Neumann's Hilbert space theory. Some physics profs tell their students to read that book to learn the Hilbert space theory they need for quantum mechanics. Generally the book uses Hilbert space, that is infinite dimensional, notation and arguments when they also work for the finite dimensional results -- cute. This approach is easier and cleaner when it works.

There is emphasis on duality, as is important in functional analysis including Hilbert space.

Some of the special emphasis is on spectral theory, important in transformations on Hilbert space.

Halmos includes some multi-linear algebra (can see again in exterior algebra and more modern approaches to general relativity) and near the end has an ergodic theorem!

At one time the book was one of three basic references for Harvard's somewhat famous or notorious Math 55 as in

http://www.american.com/archive/2008/march-april-magazine-co...

The other two references were Rudin's Principles and Spivak's Calculus on Manifolds. So, from Harvard get a good recommendation for excellence for the Halmos book.

I very much enjoyed that Halmos book and later more of his books. He was one of the best writers in math of the 20th century.

His proof of the Hamilton-Cayley theorem is a little less general than the one in, say, E. Nering's book, and I wrote Halmos about that. He wrote back a nice letter.

I learned linear algebra from a course in abstract algebra, Murdoch, Linear Algebra for Undergraduates, (about the easiest linear algebra book anyone could read), one by E. Nering, some work I did in mathematical physics, the first chapters of Nickerson, Spencer, and Steenrod's Advanced Calculus, my ugrad math honors paper on group representation theory, Herstein's Topics in Algebra on abstract algebra, Halmos, FDVS, Forsythe and Molar on numerical linear algebra, applications in multivariate statistics, some work in optimization, carefully wrote out my own notes, enough for a book, and more.

The OP list also includes Horn's book. After all that study, mostly independently and part of my career in applied math and computing, eventually I took Horn's course, intended to be an advanced course. As part of my application to his grad school, I did include a copy of the Halmos letter!

There was next to nothing in the Horn course new to me, and I didn't try very hard. Everything in the course was carefully graded. I did the homework but didn't study for the tests. On an early homework assignment, the grader made a mistake on my paper; I corrected him; and he made no more mistakes. About 3/4ths the way through the course the grader showed me the class scores -- I was first on everything by wide margins. Did that again on the final. Apparently did that again on the corresponding Ph.D. qualifying exam.

In the course, when Horn got to the polar decomposition, I got excited and blurted out in class "That's my favorite theorem" which it was and is. Horn's remark was, "Thank you Dr. Halmos".

The polar decomposition? Go through a lot of stuff in linear transformations and then discover something really simple: All a linear transformation can do is just a rigid motion (rotations, reflections) followed by stretching/shrinking on orthogonal axes. A singular transformation? One or more of the axes go to zero. In the non-singular case, the proof is really simple. In the singular case the difficulty is that there are many choices for the rigid motion part and just have to pick one.

Horn did at times have some especially good proofs.

Horn is very precise with high quality.

It's possible to have a quite fast introduction to linear algebra, but then are missing out on a lot if don't work carefully through a classic text such as Hoffman and Kunze. IIRC it is available as PDF on the internet for free.

E. Nering's book is also fine except for his material in the back on linear programming which is easy (for a lot of it, just slightly tweak Gauss elimination) but he makes difficult. For linear programming, get a good book on linear programming, e.g., Chvatal or Bazaraa and Jarvis. In the back, Nering also does group representations -- very cute.

I'd recommend Halmos, FDVS as ice cream and cake desert after something like Hoffman and Kunze.

For the numerical stuff, should look at the documentation for Linpack. Maybe also look at Forsythe and Molar. Some that has been done since then for numerical methods, e.g., for models of oil reservoirs, is impressive -- might at least glance at that.

Broadly there are two directions in linear algebra: One direction assumes that the field is the real and/or complex numbers and is aimed at analysis and applied math. The other direction assumes finite fields and is aimed at high end abstract algebra and has applications in, say, algebraic coding theory (high end error correcting codes).

The direction with real and complex numbers is, sure, a start on functional analysis (you are in a vector space, but each point is a function -- get to learn how to approximate functions by other functions, e.g., how to approximate random variables by other random variables, e.g., learn Fourier theory), but for that also should have measure theory, e.g., from Royden, Real Analysis and/or the first half of Rudin, Real and Complex Analysis. For that, as in Math 55, a prerequisite is Rudin's, Principles.

Rudin's Functional Analysis is more advanced, still. I got accepted to grad school at Brown's Division of Applied Math (didn't go there), and Rudin's FA was popular there, but I haven't seen a lot of popularity for it elsewhere.

A huge fraction of applied math, somewhere on the way to implementation and code, boils down to linear algebra.

Oh god, I still have nightmares about Herstein. I took an undergrad algebra class from a very old-school prof who thought it would be good for undergrads to be exposed to some of the problems - they gave all of us quite a bit of trouble.

Herstein is a bit obscure for a first course. Once I was pushed into a course from Herstein. At first it was in group theory below what I'd already done. I asked the prof if he was going to get to group representation theory. He said that that topic was "deep". I told him it was my ugrad honors paper. The course was beneath me, and I saved time and didn't go. For the final exam, he gave me an oral. He asked me something about Galois theory, which is in Herstein and which I learned from there for his exam. His other question was about duality. He wanted me to show that the dual space of the dual space is isomorphic to the original space. That's true and easy to show, mostly just pushing symbols around, and the notation can get a little confusing, but is not true in some cases.

Some point set topology courses consider duals of duals, and when there is equality say that the original space is reflexive. Then they want to define the weak topology and the weak* topology, etc. Can see some of this in, say, G. Simmons, Introduction to Topology and Modern Analysis, super nicely written book although I have never seen any applications for it.

Pushing around duality in linear algebra can get to adjoint transformations. To me it turns out to be a lot of careful notation for something that turns out to be nearly trivial.

Yes, on problems in Herstein, there was one on rings. It took me a while but I got it. I showed it to the prof, and he sent me to another prof who thought that solving that problem was good. He gave me a more difficult related problem, and I made a mess out of it. The problem needed transfinite induction, and I was supposed to understand that but didn't understand it very well. So, I got pissed off: I should have been studying stuff I didn't know, e.g., transfinite induction, instead of wasting time in the Herstein course that had, say, only Galois theory I didn't already know.

At that school same thing happened in a topology course: The book was Kelley, famous. Heck as an ugrad, I'd taken that book and given lectures once a week to a prof and did a stack of the exercises. I did all but the chapter on compactness at which time I dropped everything to finish my honors paper. For me to sit in that topology course was stupid.

I went to the department chair and tried to get into some material I didn't know but wanted to, especially differential geometry for relativity. He told me no.

So, I did my teaching, started violin, met my wife, and left for a good job!

> The reason it's not on the list is I just didn't see what it had to offer that isn't covered in later books.

It's concise and readable. Every word is used for a reason.

Have you checked this[1] one out?

It's free! Problem solutions are also free.

[1] http://joshua.smcvt.edu/linearalgebra/

The books in the list are pretty advanced. It would be nice to see some sort of "star" rating for each book to indicate which books the authors found best.

On a shorter note, here is a four-page introduction to the basics of linear algebra: https://minireference.com/static/tutorials/linear_algebra_in...

I studied under the author of this book [1] and I can anecdotally say that I finished with a very strong understanding of the subject. Learning linear algebra and being able to whip out some of the less common theorems has been very valuable to me as someone who uses numpy, pandas and tensorflow regularly.

[1]: https://www.amazon.com/Linear-Algebra-Applications-Otto-Bret...

I am a big fan of Hoffman and Kunze. I also think many would benefit from reading a book on ordinary differential equations after they read a linear algebra textbook. Much of the elementary theory of linear ODEs consists of combining relatively simple observations with linear algebra. Coddington and Levinson is the one I used for a course in college, but it is quite detailed. I am sure there are less ponderous options available, but I don't know any off the top of my head.

The Lipschutz (that is quoted) is good. While looking stupid and repetitive it is giving very good basis.

Applying linear algebrae to grok more complex problems can be done with the Mc Grawhill book on Mechanic (solid & punctual). It is the same pattern : simple explanation and a lot of exercises.

Thermodynanmic, quantum mechanics, relativity and electromagnetism are introduction to even more interesting algebrae.

Maybe Feynman physics lessons are the right books :)

My favorite LA books are Linear Algebra by Friedberg/Insel[0] which is a combination of Axler style book with more computation oriented one (Terry Tao has a set of lectures based off this book). Another one I like is Modern Intro To LA by Henry Ricardo[1] which implicitly introduced me to Replacement theorem which is really overlooked in a ton of LA books. Again, this book's a rigorous mixture of both theory and computation done very well. If you've never seen higher level math before, there's Linear Algebra: Gateway to Mathematics by Robert Messer[2]. It has tons of commentary about elementary set theory and proof techniques along the way. Whenever someone mentions Axler's book, someone else brings up Treil's book. But there's a third one in the same league/group which is Linear Algebra: An Introduction to Abstract Mathematics by Robert Valenza[3]. Other favorites are Coding the Matrix by Philip Klein[4] for Python aficionados and Linear Algebra Through Geometry by Banchoff/Wermer[5] for those who like geometry.

If you are way beyond all this, you can still pick up new things from Advanced Linear Algebra by Steven Roman[6].

[0] https://www.amazon.com/Linear-Algebra-Stephen-H-Friedberg-eb...

[1] https://www.amazon.com/Modern-Introduction-Linear-Algebra/dp...

[2] https://www.amazon.com/Linear-Algebra-Mathematics-Robert-Mes...

[3] https://www.amazon.com/Linear-Algebra-Introduction-Mathemati...

[4] https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...

[5] https://www.amazon.com/Algebra-Through-Geometry-Undergraduat...

[6] https://www.amazon.com/Advanced-Linear-Algebra-Graduate-Math...

+1 for counting from zero

Seconding Klein's Coding the Matrix. It's a great alternative introduction and actually implementing linear algebra ideas in Python is both useful and a great aid for learning.

The auto-grader is helpful for making sure you got the right solutions.

Higher Algebra by Hall and Knight

Reply on news.ycombinator.com