No-bullshit guide to linear algebra | Hacker News Reader

No-bullshit guide to linear algebra

gumroad.com

185 pointsIrishsteve66 comments

I always have to mention my all-time favorite introductory book on this subject:

Liner Algebra Done Right (http://www.amazon.com/Linear-Algebra-Right-Undergraduate-Mat...)

Its breakthrough is its focus on the basic algebraic properties of vector spaces and linear maps between them. It de-emphasizes matrix computations and especially determinants (they are covered, but only insomuch as they are necessary).

In my experience, the result of a typical linear algebra course is most students don't fully understand the determinant and more importantly they don't understand the proofs of major theorems which involve long manipulations of the determinant. They also don't understand the more algebraic side of the subject because they aren't given a chance to--it's not covered in much detail. The result is they don't understand the subject overall much at all.

This book is based on the observation that the abstract algebra involved in linear algebra is actually remarkably easy, much more so than arcane determinant manipulations.

That particular series contains some of the best math texts I've read. My instructor used the Abbot text ( http://www.amazon.com/Understanding-Analysis-Undergraduate-T... ) for our real analysis course. The books are very aptly priced and are extremely well written.

This, a thousand times this.

My first introduction to Linear Algebra came from Strang's OpenCourseWare videos and, a year later, a class that used his book. Despite having gone through the material twice, I struggled to apply it to actual problems. Connecting the real world (err, models of the real world) to the blocks of numbers was the sticking point. What are the matrix's units? How do I write down a matrix that does what I want? I looked things up and prayed that whatever black magic the mathematicians on wikipedia had invoked in their derivations would match the implicit assumptions I knew I had to be making.

Then I was forced to re-take linear algebra in college. We used Axler's book and suddenly everything made sense. The blocks of numbers that you type into matlab are simply shadows of the mathematical concepts of a vector and linear transformation. The abstract mathematical concepts are actually closer to the real world concepts they model because they don't have to depend on arbitrary choices of coordinate system, etc. Once you understand how to model real-world quantities with abstract vectors and linear maps (easy) and how to translate those coordinate-free equations into coordinate-dependant matrix equations (easy), THEN you understand the material enough to apply it without fumbling around trying to paste other peoples' formulas together (hard).

OP's book looks well written but it entirely follows the tradition of emphasizing the blocks of numbers over the abstract concepts. Linear maps don't appear until page 75, while blocks of numbers enter on page 5! That's nuts! Use Axler.

This book treats linear algebra like any other mature mathematical subject. It just so happens that it's easier to understand than most mature mathematical subjects. But yeah this is just how any pure mathematician would approach the subject (which is why I liked it so much).

I'm curious - did you work through this book on your own, or as part of a class or with help from others?

I started reading it, attempting most of the exercises at the end of each chapter, and some of the 'as you should verify' parts. I found the 'aha' moments during the text or when solving the exercises really enjoyable, but I got stuck often enough that I began to only pick up the book when I felt super-alert.

I stopped about a third of the way through, a few pages before the introduction of eigenvectors and eigenvalues. I would like to pick it up again, but am worried about how to maintain motivation the next time I'm faced with a page where I'm stuck/confused for 30 minutes.

EDIT: Thank you to the people who replied. Your empathy (~"you're not alone in getting stuck") and encouragement (~"you don't have to grok everything the first time through") have given me new motivation to try this again.

Mathematicians are chronically lost and confused. It's our natural state of being, and it's okay to keep going and take what insights you can.

I suggest you don't worry too much about verifying every claim and doing every exercise before moving on. If it takes you more than 5 or 10 minutes to verify a "trivial" claim in the text, then you can accept it and move on. It's healthy, if you get stuck on such a problem, to think about other problems or come back to them later. It's not uncommon to find that by the time you revisit them you've literally grown so much (mathematically) that they're trivial.

> Mathematicians are chronically lost and confused. It's our natural state of being, and it's okay to keep going and take what insights you can.

This needs to be emphasized more in learning maths. Once I got over this fact and was guided by my interest rather than wanting to grok everything, I actually started understanding a lot more.

Someone should tell us that sooner then. I've been trying to get over math anxiety for a long time, and having professors that say "it's simple, what don't you understand?" when you try to get help didn't help much.

A common joke among mathematicians.

Two mathematicians are working on a problem and one makes a claim to which the other replies, "That's trivial!" The pair stare at the claim for ten minutes. The other says, "Is it trivial?" They go home, think about it all night, publish three papers about it over the next two years, and then they agree, "Yes, it's trivial."

I started reading it for a class but I wasn't assigned all the problems. Later, I managed to propel myself through it by building confidence with the problems of the first few chapters (which were easy by that point) and then embracing the sunk-cost fallacy to get through the rest.

One problem stood out. I forget the chapter, but I'll never forget the problem: prove that normed vector spaces are inner product spaces IFF the parallelogram law holds. It took about half a day to crack that problem and I was completely lost for about 90% of it. (That's not a strategy. The first paragraph was my strategy. My only point here is that you aren't alone in getting stuck.)

I actually skimmed the book over before I took my first course in linear algebra (doing some of the exercises on my own, but certainly not all--I skipped anything that seemed too hard or just too boring during this reading). At that point, there were definitely parts of the book that were challenging to me, and I just jumped over them.

Then when I took the actual course, I more or less ignored the main course textbook and instead tried to tie whatever happened in the last lecture with what I found in Axler.

I considered Axler my secret weapon. It honestly felt like I had access to secret insights that trivialized the class and granted me the same intuition the professor had.

I agree. I am also working through this book and feel the same as you. I think also part of the problem maybe not being able to discuss this with other people. I'm trying very hard to get through it but I think I miss being able to discuss problems with other students in a traditional classroom setting. These sorts of discusses can really help clarify things rather than just staring at the page for a long time.

A side note: how do you feel when you've lost your motivation? Is it common to barely be able to keep your eyes open? Or is that just me!

Not just you. I'm procrastinating on one of those too-damn-hard math problems right now!

It's not just you. It's probably easier to accept that I'm sleepy and should take a rest, than to acknowledge that I need to focus and think harder about the material.

I do so agree. Axler's book is also by far the most Geometric book on Linear algebra that I've ever read.

Oh yes, +1. Linear algebra is about vector spaces; matrices are a separate topic.

Unfortunately, Axler does not define determinant as a volume scaling coefficient and goes the still arcane way of “(-1)^\deg…” which does not really do justice to the concept.

Check out Sergei Winitzki's “Linear algebra via Exterior Products” https://sites.google.com/site/winitzki/linalg It's free for download and treats determinants in a more geometrically inclined way. BTW, it's written by a physicist, not a mathematician.

No, he defines the determinant as the product of the eigenvalues, and this is almost the entire point for his writing the book!

Any idea how this compares to Hoffman-Kunze? I liked Hoffman-Kunze because, despite being dense, its no-nonsense and very rigorous.

What is the equivalent book for other topics in math (I'm trying to self study) e.g. Calculus, Abstract Algebra, Probability, Statistics?

Nice title.

I'm hoping for a Math textbook written like its an abusive drill sergeant.

"You think matrix inversion is hard, huh? We're still just laying the mothafuckin groundwork, you halfwit. You ain't BEGUN to see hard yet.... You have no CONCEPTION of what a hard math problem might look like. This math is not hard. This math is easy. And I'm gonna make you bust your brain against these exercises until you make it look easy. Now pick up that goddam pencil and WORK your brain."

Wow. I'm glad I didn't take the Math courses you went through :) Perhaps, that's why I like it so much.

That's because you're a pussy.

I'm a teacher. My colleague is a former marine sergeant with a Computer Science degree. We'd soon make a ma(thematicia)n out of you.

My math teacher was an ex-narcotics cop. I was a bad student. I could do it, but I had a serious (albeit earned) attitude problem and no motivation or capacity to pay attention.

Currently, PhD in statistics and nearly a qualifed actuary (I've done all the maths bits of the qualification).

You are doing Spaghetti Monster's work.

how do you earn an attitude problem...?

You gotta fight every single fucking day, so you don't get beaten up, and your world view and how you let the world view you are going to change. I think the teachers in my school could have maybe done anything at all to prevent this.

I'm gladder still that I wasn't your student. You can't get creativity or love from bullying, only slavery.

P.S: I'm in a top CS Grad school, so no, thank you, for your offer of consdescension. I'm sure, you and your man-ly pals will have a happy time making more soldiers than mathematicians.

One of the harder exercises I had attempted in my linear algebra course was the "connected tanks of water and salt".

After searching for the solution, I found this awesome resource that combines linear algebra, differential equations and graph theory to model salt flows in various networks of tanks.

http://www.karlin.mff.cuni.cz/~slavik/papers/mixing-problems...

>"connected tanks of water and salt"[...]After searching for the solution

hehehe

That's a bit beyond linear algebra -- it's only soluble (no pun intended) using linear alegbra's methods for any arbitrary instant of time. The full evolution of the system over time requires classical analysis and differential equations having nonlinear properties.

My point is that the linked article uses linear algebra as part of a problem statement that requires more than linear algebra for its analysis.

Author here.

Thanks for all your comments!

This is the followup book from the No bullshit guide to math and physics and is written in the same style. I'm still doing some final touchups on the prerequisites chapters (Ch1 and Ch2) and I want to beef-up the Applications chapter, but the core material is done.

Several of my students asked to see the book---even unfinished---because they wanted to use it to study for their finals so I decided to make a pre-release. Once I finish the Applications chapter I'll create the print version and a nicer looking eBook (sans-serif). The current version is intended for easy printing at home of sections that you have to study for.

@Irishsteve thanks for posting! I am on a plane right now going to SF (tourism mostly, but if anyone wants to talk textbook business... hit me up) and I couldn't resist the $10 for 1h of internet time. I had a nice surprize when I checked HN!

Page 3 - phased -> fazed (in figure explanation)

Thanks!

I know you point out it's unfinished, but I hope you've got a good team of editors, because there are a lot of grammar and spelling mistakes. The content looks awesome though - look forward to the full thing.

Very nice. FWIW, the table of contents links did not work for me. (Firefox's .pdf viewer under Ubuntu, if that helps.)

60 pages before defining vector spaces seems like a not so good approach to linear algebra. Since Linear Algebra Done Right has already been recommended, I will suggest Linear Algebra Done Wrong: http://www.math.brown.edu/~treil/papers/LADW/LADW.html

I know, I would have liked to have linear transformations and vector spaces moved forward, but without the computational skills (grunt work) you can't really do much with vector spaces and lin. trans. except define them.

Hence the current ordering of chapters...

Then just define them! The entire subject is about linear transformations on vector spaces. The biggest problem with linear algebra as its taught today is that students uniformly come out with an understanding that linear algebra is about matrices. It should be made clear that a matrix is simply a representation of a transformation, and then when you do teach the grunt work operations, you can teach them as what they actually are (multiplication: composition of transformations, determinant: area of unit square under transformation, etc.)

Khan does linear algebra: https://www.khanacademy.org/math/linear-algebra

For those that like exposition in their texts:

http://joshua.smcvt.edu/linearalgebra/

I wonder what the author considers bullshit.

Probably all that rigorous stuff when it comes to proving some theorems. Its a practical book.

That's unfortunate. Linear algebra is one of the few mathematical fields that one can really benefit from knowing the theory.

Author here.

Any material that is placed in a textbook to inflate the page count and legitimize the $100+ price of the book.

Actually, the forces of bullshit are strongest in freshman calculus and physics texts; linear algebra books are much better on that front.

Apparently (unless I missed it) the Singular Value Decomposition. Which is unfortunate. Otherwise, it reads very reasonably.

SVD is in there, in the "matrix decompositions" section.

aybe I should make it more prominent.... but usually for UGRDs they focus on eigen-decomp so I put the main focus there too...

My last math class I ever took was plane trigonometry in college. Our professor was Russian, a genius, and a total jerk. He hated that he had to teach us lowly morons. He told us, on many occasions, that we were "stupid Americans", that "Russian children [were] smarter than [us]", and that he would "give us below an F if [he] could." Really managed to ruin what little interest I had left in math. I can appreciate a book that gets right to the meat of the matter. But for my personal relationship with math, I need something that wraps everything up in a warm fuzzy blanket first. Major props to the author for not wanting to inflate page count with crap, though.

Paul Lockhart's "Measurement" does this (the fuzzy blanket) and covers all of geometry, trigonometry, and calculus!

Thanks, I'll check it out!

I don't know how Matrix computations wasn't mentioned. When it comes to practical implementations of linear algebra concepts this book is your best friend. [1] http://www.amazon.com/Computations-Hopkins-Studies-Mathemati...

This looks like a very readable take on linear algebra - the kind of book that you could leave with a really firm understanding of how all the bits fit together.

It looks theoretical and very deep - not what you need for passing an undergraduate math course. Maybe exactly what you need to pass a graduate level statistics course.

It is not very precise but I am undecided if this is a good thing or not.

Right. I would like to somehow subscribe to new and up coming book releases and their reviews. You know, submit a link of this book on a site, let someone else take the risk of buying it, then get updated on the first review and on the average reviews after a week.

Note: The author should add a: "I may want this." field with an email subscription option :/.

Seems Good One. Till now, Linear algebra by Gilbert strang is my all time favorite. http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

This guy gave me this book before my final and it helped me pass. Heavily recommend!

Nice title, lousy default-width typesetting; nobody has an e-reader that will fit this, and it would look equally bad in print. Also, it's 'formulae' not 'formulas'.

I know. This isn't the final version, actually the width will stay the same but the paper size will be 5.5in x 8.5 in (half of letter size).

For the eBook, I'm targeting tablets mostly (see http://cnd.mcgill.ca/~ivan/miniref/noBSmath_preview.pdf ), but this is not ideal for smaller screens.

I have a kindle forma as a work in progress, but it's not idea as some equations really need a wider screen, so not sure how to break them up...

Long term, I'll have to figure out how to generate clean epub with math as images (I would prefer MathJax, but not all ebook readers support js). That should fix the small screen format.

Thx for your feedback!

Will there be a print version? I have the print version of the other no bullshit guide and love it. I much prefer print for this kind of material.

Seconded

From the text layout of the pdf it is anyone's guess. There are no verso/recto margins but chapters always start on odd pages and the header/footer is setup for verso/recto. Hopefully the final version will have better typesetting regardless of distribution medium.

Yes print version is on the way. I will use the proceeds from the pre-release sale to hire my trusty copy-editor again, work on some exercises, and then we'll have a v1.0 release, print and PDF.

Any chance for electronic formats other than PDF? The lack of real reflow in PDF means it is pretty useless on smaller devices. Epub or mobi are equally useful, anything besides just PDF really...

I've been learning how to generate .epub, but I'm having trouble with generating the math equations that render nicely.

Do you know of any epubs with math in them that I could imitate?

Would you be willing to try out some of my .epub testing and give me feedback? If so please send me an email.

will try it out to see if there's anything in store for a beginner in Machine Learning !

I look at the title of the book and I think "amateur hour".

;)

Well then the title worked! The idea is to disguise a serious textbook that teaches the material and understanding, but not make it look like a non-serious text so as not to scare off the reader.

Reply on news.ycombinator.com