Immersive Linear Algebra – A free interactive online book | Hacker News Reader

Immersive Linear Algebra – A free interactive online book

immersivemath.com

434 pointssamlittlewood78 comments

This looks awesome. The popup-help's are a little jarring though since there's no indication as to which words will trigger a popup. Also:

"Oh come on, you should know what a vector, v, is by now. Check out Chapter 2, for crying out loud."

seems kind of harsh to say to the reader while they're reading the first sentence of Chapter 2..

"Kind of harsh" is an understatement. Insulting the reader like this is not acceptable. Mathematics is already difficult enough to learn by its very nature. Authors, you don't need to help make it more difficult. We already have a hard enough time convincing people that they can learn something this difficult.

Thanks for the feedback. We have removed the popup for "vector".

I always see linear algebra on HN and many people comment on how they never understood the subject. This makes me ask: what exactly is it that people don't get about linear algebra? What makes it appear to be a difficult subject?

As someone who has used linear algebra almost every day in some form over the last decade, it's hard to get a perspective of what aspects are challenging to the beginner. And since I TA courses that involve linear algebra, it is good to know where the problems are.

What it is, I think was my major problem.

Matrix multiplication, ranks etc. are some of the most random seeming operations you can tell a student to memories.

My view is that the people who get to matrix algebra didn't have problem understanding the random seeming operation of previous math courses. Consider the existence of the "FOIL" mnemonic for multiplying two binomials (https://en.wikipedia.org/wiki/FOIL_method ), or SOH-CAH-TOA, where it was easier for me to just remember the definitions than the acronym. I think those are hints that some people consider those to be 'random seeming'.

When I took algebra, I remember being tripped by all the new vocabulary. Why do mathematicians say something is "Abelian" instead of 'commutative'? Why is it important to have terms like "group", "ring", and "vector space"? Sure, I learned the definitions, but at that level they were 'random seeming'.

I think it's mostly because mathematics is taught as is with little or no attention to practical applications of the theory. For instance, till date, I could not see how eigen values/vectors are useful.

One nice application is that the eigenvectors of a stochastic matrix correspond to stationary states of the Markov chain. That leads naturally to a conversation about PageRank, which is super useful because it made billions of dollars for some people. Then you could talk about the use of eigenvectors for principal component analysis in data science, which is making a lot of money for other people right now. If these examples are not useful, then I don't know what is.

Thanks, the application to PageRank looks interesting. I shall resurrect my knowledge on eigenvectors and eigenvalues and make some sense out of it.

Interesting. Eigenvalues and eigenvectors are one of the most fundamental applications of linear algebra. In physics, quantum mechanics is a giant eigenvalue-eigenvector problem.

One application I really like is in machine learning: the eigenface algorithm.

That might all be the case. I would simply like a thoughtful introduction to the topic about where this abstraction comes from. What is the idea behind it and what was the motivation for its invention?

After I understand the plain English concept, then give me the math notation and proof and applications.

The way it works today is often to simply leave out the first part and I believe this is why many people find it hard to develop intuition and a real understanding of the concept.

For man it's just there, you memorize it, you apply it, you take the result and simply have faith that it is what you need because some book/ prof said to solve problem X use eigenvalues.

The simple idea is that while linear maps(matrices) are generally a combination of rotations, scaling and other stuff in higher dimensions, sometimes they will act on a vector simply by scaling it.

Since scaling is a really easy operation to understand, the space generated by these vectors will be really easy to understand. Note that different vectors can be scaled by different amounts.

Now it often turns out that the space generated like this is actually the whole of the space under consideration and this really simplifies the linear map we started with. Hope that helps.

I found that the level of abstraction that linear algebra is taught at is really weird. On the one hand it was never really taught in connection with algebra in general, or put into a wider mathematical context. On the other hand we were never really shown any real world used of linear algebra or how the concepts maps onto practical applications.

It wasn't before years later when A) I'd taken more abstract algebra courses that introduced concepts like tensors and fields and B) I'd taken more practical computational courses where I had to do things like use linear algebra in 3D graphics and use eigenvectors to do dimensionality reduction and PCA, that I really understood the subject and its place in the world.

Just saying here's a random object (that we'll call a "matrix"), here's some random steps (that we'll call "taking a determinant"), now memorize how to apply the steps to the object and see if the result is zero, didn't lead to much deeper understanding.

I think what courses often fail at is to specifically mention that all of these things are abstractions invented by people to deal with a specific issue and then WHY IT HELPS solving this issue.

Whenever I studied Linear Algebra it was simply "here's a matrix, here is some algorithm, use it to get some number seemingly coming out of nowhere and just believe us that this number is what you really need."

Note that I am not immediately interested in mathematical proof of why this method is correct. I want a plain English explanation of what is going on but instead you usually just get a bunch of notation to digest.

I see that chapter 10, on eigenvalues/eigenvectors, is "Coming soon!".

Eigenvectors never made intuitive sense to me. As with the various decompositions (Cholesky, LU, etc), I could apply the math as algorithms to follow, but never got to the point where felt I could apply them to new problems.

Then again, in practice, I've only needed eigenvectors once since college, and it was more a rote implementation described in a paper. (In other words, don't feel like you should educate me on them here.)

Since you are a TA, don't you get some idea of where your students struggle with linear algebra?

Interesting. I used linear algebra with quantum mechanics where eigenvectors represent quantum state of the Hamiltonian which is how I initially understood them.

And as for TAing, have you ever TA'd? You definitely get a feeling but I can count numerous times when I've stood in front of the tutorial class and asked if there are any questions only to get no response back. I think it's a symptom of first years. I've TA'd calculus as well and I get similar responses. It's very frustrating sometimes.

I have TA'ed. Asking for questions in public doesn't help much. Students rarely like to admit they don't know something.

I was thinking more of when you look at their assignments. There are often multiple ways to approach a problem, and the route chosen can reveal something about one's level of comfort.

It doesn't help that university TAs get almost no training in how to be a TA. At least, I didn't.

Yeah I know. It's hard to squeeze information out of them. I actively tell them to please, please come to me if they don't understand something and I have office hours for a reason. Yet no one takes advantage. At a certain point, you can only do so much since I'm a grad student and not a lecturer and my time is finite.

The problem I've found with assignments though is that people copy and cheat. Many times someone will do very well on assignments and then do absolutely terrible on midterms and finals. It's very frustrating. I remember one course where everyone did nearly perfect on the assignments and yet the final and midterm followed the standard bell curve.

It's worth devoting a portion of classroom time (in my opinion, a substantial portion) to discussion of the topic, perhaps focusing on particular problems and generalizing from there. For example, you might select random students each day to present problems from homework. If you've ever noticed that you learn something better once you teach it to someone else -- well, it works for your students, too.

Moreover, if you create a non-judgmental environment in which people are free to talk about their approaches to problems and get feedback not only from you but from other students as well, then just by watching carefully, you will learn some of the more common gaps in understanding. (Note that some students will not talk in these situations unless forced, but that does not mean they do not benefit from following the discussion.)

If you're anything like I was when I was first TAing courses like this, you might think that if you do this, you won't have enough time to "cover the material". But I put it to you that a lecture that is not absorbed doesn't cover anything.

> You definitely get a feeling but I can count numerous times when I've stood in front of the tutorial class and asked if there are any questions only to get no response back.

When I realized I was understanding subjects better than my classmates, I'd take on the task of asking the "dumb" questions for them. It was partly selfishness, I was tired of answering those questions for them outside of class. But it really did prove helpful to my classmates. It really helped that, when I was sitting with the students, I got to hear them mumbling and grumbling about what they didn't get. So I knew exactly what questions to ask to get the professor/TA to help my classmates.

I never TA'd myself so I have no idea if this would actually work, or at least work consistently. But, if you have a couple students that really seem to be getting the material, you could try talking to them one-on-one and ask them to help you out in this manner.

Linear algebra (and most calculus) didn't really click for me until I needed to apply it to physics problems.

The general idea started to make sense in mechanics class when I could see matrices are convenient shorthand for solving multiple variables at once, and which behave like 'regular' variables when trying to manipulate them algebraically.

Eigenvalues and eigenvectors didn't make sense until quantum mechanics, where your 'operator' is effectively a matrix and your 'state' is a vector. The allowed observed values are the eigenvalues, and your final state after measurement is the corresponding eigenvector. Wave functions are then an extension of this from discrete space (useful for modelling spins for instance) to the continuum of position space.

I love eigenvectors. For me, the rest of linear felt like a bunch of rote rules and algorithms for figuring out vaguely-interesting stuff. And then suddenly I find out that there's this amazingly non-obvious but remarkably powerful structure hidden inside matrices that I'd never even been aware of before.

Quantum mechanics is a huge application of eigenvectors, but I also really enjoy things like the "moment of inertia tensor", whose eigenvectors are the natural axes of rotation. Or better still, coupled oscillators, where the eigenvectors give "normal modes" of vibration. (And if you look at coupled first-order differential equations, eigenvectors can tell you all sorts of things about "trajectories" of the solutions. There are great applications of that to things like population dynamics in biology.)

I enjoyed this articled called "Principal Components Analysis for Dummies" that illustrates eigenvectors and eigenvalues. https://georgemdallas.wordpress.com/2013/10/30/principal-com...

There's also this interactive page for eigen- vectors and values: http://setosa.io/ev/eigenvectors-and-eigenvalues/ (as well as a sibling for PCA: http://setosa.io/ev/principal-component-analysis/)

The point of an eigenvector is that its direction is stable when you hit it with the given transformation. That is, it's an invariant direction of your transformation. This is analogous to the derivative of an exponential function being a simple multiple of that function. It makes it easier to reason about the action of the matrix, and simpler to represent.

Imagine I have a matrix transformation that takes a given input vector and converts it into a superposition of a thousand other vectors in random directions; that's far harder to reason about than if if just kicked it farther or brought it closer in the same direction.

I took LA in university and I really wish they had offered it in high school or even middle school. It was a requirement for my major but not a requirement for any of my other math courses so I treated it like an elective. It was only after I took the course and started applying it that I realized how applicable it was to almost every math problem.

Instead of calculus for business majors or art majors they really should be using LA instead.

At the moment I'm trying to teach my son LA but I can't find any books for 8-10yo

In almost every case, confusion arises when the "how" is taught with no explanation of the "why". Why does that formula work? Why did you use that formula?

Whole worlds were opened up to me when I learned that the fundamentals of trigonometry, like the simple computation of sin of an angle, are all related at its base to a square but no math teacher ever made that connection for me.

The other aspect that I found with LA compared to almost any other math subject I took was that it was almost entirely mechanical with little to no connection between my understanding of the subject and my grade. The exam questions where mostly along the lines of "find the eigenvector" and "find the determinant" rather than "what's an eigenvector" and "what's a determinant", so I got good grades despite not having any idea how to answer the latter questions, or really having any idea what I was doing. Understanding was neither necessary nor encouraged.

This, exactly. I could find an orthonormal basis or an eigenvector, but I never quite wrapped my head around their importance, especially visually.

What's challenging to me is my inability to develop intuition about LA simply because most of the time I have no idea what (the purpose) I am doing.

How does one build intuition about LA?

At my (American, liberal arts, flyover country) university, linear algebra was my first real mathematics course. (The prior classes were calculus and a "proof" class, AKA intro to intro to number theory.) I'd been introduced to the determinant, Gauss--Jordan elimination, and so on, in high school, but all in a very mechanical way.

I think this transition between mechanically generating solutions to problems using pattern matching and some basic algorithms on the one hand, and the more mathematically mature approach of exploring problem spaces using pattern matching, some basic algorithms, and intuition can be difficult for many people.

It's not necessarily the material -- you can get used to almost anything, and even convince yourself it's easy or obvious with enough familiarity -- but the lack of intuition. When you're first learning linear algebra, its fundamental unity is not obvious, especially if the instructor does not take pains to point it out. (And even if the instructor does take pains to point it out -- well, it's hard to understand why the instructor is saying we could do this computation this way or that way.) So in the absence of existing intuition or any perception of unity, linear algebra becomes another target for pattern matching and basic algorithms.

As it happens, I've never, ever felt like I didn't "get" linear algebra. However, I almost always feel like I "get" it now and all my prior conceptions of it were a confused muddle.

It's a global vs sequential learner thing. Linear Algebra is usually taught as a giant pile of tricks, techniques, and terminology. This is tractable for sequential learners like myself - I really enjoy picking up new tools in a narrow context just for it's own sake. Highly global learners basically can't learn anything this way.

So, in short, my advice is "add more context."

I never studied math in college, in fact I barely studied math in high school. It was always daunting to try to parse the notation and figure out how the abstract symbols corresponded to some piece of reality.

When I became a self-taught developer I found my math skills continuously lacking. I started teaching myself on Khan Academy and really picking it up a lot better because of the simplicity of the language and the good examples. I finally realized I learned math best visually.

Interactive lessons like these are great. There are things that can be improved about this book (load times and enhanced interactivity) but all in all this a great resource for people that learn best visually. I'll come back to this soon in my future self-education.

What exactly do you do when you need math skills in developing? What do you develop with it? Some specific examples?

I've done math while developing 3D stuff, physics simulation and a lot of things - yet I never felt like my math skill mattered that much. And I really dislike theoretical math like in school.

Math is certainly not for everyone, but I actually found that once I started diving into pure math (NOT what was taught in my school's math curriculum) it was a whole new world.

Did your school actually teach pure math?

I bet a lot of programmer types had similar frustrations to the computationally intensive version of math taught in many high schools when they knew that these processes could simply be automated.

This. I had the same experience. By the time I realized this, I was sufficiently jaded that it made devoting my own time to learning it sting more than a little.

I don't even know what pure math is to be honest. The courses I specifically lost interest in was linear algebra, discrete mathematic and statistic-something. I did however use linear algebra just after while writing a simple collision/physics engine prototype and found that to be very fun.

I haven't had to rely a lot on math skills for a large portion of my career. For most "full-stack" stuff I've been involved with, it barely gets any higher than algebra. However, I've found math creep up though here and there, whether it be in understanding how projections worked in the mapping software I wrote, the computer graphics code I assisted with, and also in my routing algorithms for traffic flow.

Nowadays my lack of knowledge is most apparent in studying machine learning. There have been some very high level concepts employed, at least for me: Jacobian matrices, partial derivatives, tensors, etc. I've been able to fake that for a while, the algorithms get abstracted in the libraries. Yet the abstraction obscures a real understanding for me in how they work.

I'm in a similar situation, not getting much math in college while pursuing an unrelated degree and only teaching myself programming later. I've subsequently addressed my math deficit through extension courses at local universities, Coursera and Udacity MOOCs, Khan Academy, etc.

There two areas where I've felt the need for advanced math most keenly during my software development career are: 1) Similarity scoring in information retrieval and recommendation engines. 2) DSP.

I'm sure that this a wonderful book, but please note the error right off the bat in equation 1.3 The tangent of the angle in question is b/a NOT a/b as stated. Probably a good idea to keep an sharp eye on the math as you go along in this thing.

Thanks for the catch! The site has now been updated.

Send me your name to immersivemath@gmail.com and we will add you to the acknowledgements section!

That's o.k. I'm usually the one making boo-boos like that. Glad we caught it.

This is actually pretty cool! I spent the last few weeks going through Gibert Strang's popular OCW course[0] & I'm sure this would serve as a great companion. I can't wait for the chapter on Eigenvalues to be published as that is something I don't yet grasp intuitively. Great work and thanks for making this free and accessible!

[0] http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

shoutout to LAFF as well

- https://www.edx.org/course/linear-algebra-foundations-fronti...

- http://www.ulaff.net/

might be good for you if you're motivated by a deadline, graded progress, some Matlab/Octave implementation, and certificate of accomplishment, vs. self-paced Strang (AFAIK).

LAFF is great

Had no clue about this course! Thanks a lot for the recommendation!

Similar project, but for multi-variable calculus:

http://mathinsight.org/thread/multivar

Is this available in an off-line bundle? I am becoming increasing wary of online books/training/applications that can not be read locally. If I am going to take the time to read through a full book (possibly weeks of reading) I want to be able to use/reference it in 5 years like my paper books.

most of the value of a good math book is that years after reading it you can use it as a reference to look things up you will inevitably forget.

While I don't disagree with your comment, I think once you've learned this material you could reasonably look it up in a different book. Put another way, going through this course might greatly increase the value (to you personally) of having any other linear algebra text available. It's not necessary to look things up in the same textbook you learned them from.

This issue has come up with MOOCS also. If you look at the MOOC sites they offer there class content including videos as downloads. Even tho most of the MOOCS are introductory courses and you probably would need to get a good book to go along with them lots of people were requesting downloads of the content. There is a variety of reasons why people want off-line bundles. Its just something I have gotten use to with this sort of content.

In terms of programming and linear algebra - please consult someone who is actually knowledgeable about the subject if you're implementing it in code.

Linear algebra without error analysis is very dangerous. Many many things are theoretically useful, but can't be used in practice. You can't calculate determinants, you can't count unique eigen values, you can't use certain decompositions.

Unfortunately this isn't really topic you can do a quick tutorial on and start writing new algorithms

and also don't compute the inverse if you don't have to!

Slow to load (possibly due to HN traffic), but once it does, it seems like it's got the makings of a great learning tool. Waiting for matrix chapter since that's where I stopped learning Linear algebra on both my past two attempts (Gilbert Strang made my mind explode as I tried to comprehend past 4 dimensions. And then I just got lazy). Really want to pick this up because without linear algebra it's easy to get lost in all the major streams of Machine Learning. At least that's what I felt when I tried to skip linear algebra and move on to ML.

Sigh. I sometimes wish I paid more attention to my studies while I was in school instead of goofing off and playing card games :'(

At some point, I'm hoping to get enough of this to solve this problem: you've taken a picture of, say, the Mona Lisa in its rectangular frame, but because of crowds you weren't in line with dead center, instead you were 5 meters back, 1 meter high, and 2 meters to the side, and not even pointed at the center. Your photo now contains some quadrangle that is a projection of the rectangle. I'd like to tag the four corners and have an algorithm map the photo to its original rectangle - I intuit there's enough information in the photo and the four tagged points, given that the original is actually a rectangle.

This exact application appears in the Coding the Matrix class on Coursera.

http://codingthematrix.com/

Definitely doable - see the various scanner apps in your app store of choice. They work really well.

A word of warning about solutions to this problem. The "easy" version states it this way: the mapping you want takes (x,y) to ((ax+by+c)/(gx+hy+i),(dx+ey+f)/(gx+hy+i)) for some a,b,c,d,e,f,g,h,i -- a general "projective transformation". If you decide ahead of time exactly what rectangle you want to map to, you have 8 constraints on a..i (two coordinates for each of four corners) and one irrelevant degree of freedom (if you multiply all of a..i by the same constant, nothing changes) so you have "exactly enough" information to find those parameters. So you solve the equations and you're done.

This works, provided your measured positions are perfectly accurate, but it isn't the best you can do. The reason is that a single projection actually has one degree of freedom less than a general projective transformation. (Three parameters to say where your camera is located. Three to say how it's rotated. One scaling factor. Total is 7 rather than 8.) This means that there's an extra (slightly hairy) constraint on the parameters a..i, which turns out to mean that the aspect ratio of the rectangle you're mapping to is determined for you.

Imposing that constraint will give you (1) more information -- you won't have to know ahead of time what aspect ratio you have -- and (2) a more accurately estimated transformation.

You can find more details of this stuff in, e.g., this PDF document: http://people.csail.mit.edu/bkph/articles/Harmful.pdf .

HN previously had some discussion on an article that used IamgeMagick to do this correction on a video: https://news.ycombinator.com/item?id=8713070

I was thinking of working in a lab that does research in the field of computational biology. However, I never took a linear algebra course before so I always felt like it would be a waste to make an attempt. I did a quick skim and this looks very promising. If I can comprehend this, then maybe I will be of some use in the lab. Thanks for sharing :)

This looks amazing and well timed as I've been attempting to learn Linear Algebra on the side. However, everything after Vector Products is "coming soon". I will definitely use this once it's all there, which i hope is soon. Heck I'd be willing to pay for it if it were ready now.

Is this available in an off-line bundle?

At this time, it is not available as an off-line bundle.

Really great site. It's clean, look forward to seeing the complete interactive book.

Does anyone know if there's source code or a github page associated with this? It would be nice to submit fixes to errors, typos and maybe make suggestions.

For starters, there might be a way to do away with the loading screen for shorter pages.

At this moment there is no github available, but bugs reports are welcome at immersivemath@gmail.com!

This is the first time I've ever heard someone say algebra with a hard g sound.

https://www.youtube.com/watch?v=TIcSnsmitHw

Thanks for this! I am currently taking linear algebra with Serge Lang's book and I feel that this is a good addition to my studies. When do you expect to have the other chapters done?

Cool, well doesn't work on my mobile device, but I'll definitely check this again later since I need to review more Maths before going back to uni

Not Free? That's fine, but I couldn't see a statement of that. If so, I wonder how they plan to do DRM.

This service is free of charge!

Are you involved with the project?

If so, thank you. What is the license?

Are there any ways to contribute?

To bad that teachers rarely have the freedom to teach using tools of their choosing

From a technical aspect, this is an excellent implementation of MathJax.

70 seconds to load a chapter? That's a terrible benchmark for even for some of the heaviest websites out there!

This may not be a popular opinion but I (and many ordinary readers like me) see that link as a website. Not a book.

It feels heavy and overwhelming to see a large number of 3D diagrams and visual depictions on just one web-page. Having to scroll down to read the full chapter with all that animation and "motion" is probably a bad move too. Given that this is supposed to come off like a book you can probably ditch the scroll.

Ideally, you'd want to give away few concepts in small easy-to-understand chunks with just 1 or 2 figures per page. And let the reader flip/click over to the next section like it happens with an ibook or kindle book or even a real physical book.

IMHO the idea of ripping apart a book at its spine and forcing the loose design of websites over it is a complete no-go for avid book readers. Especially for the mobile and tablet users (probably even for the desktop users!, why else would everyone insist to download PDF, ePub or other artifacts?). But I'm sure that a section of developers over here wouldn't agree with my opinion. So take it all with a pinch of salt.

Also just the place where I'd let the designers take over.

> 70 seconds to load a chapter?

The full note says "up to 70 seconds on tables." On my computer it took less than 2 seconds.

It took over a minute on mine; but I'm sure they're hammered right now.

Reply on news.ycombinator.com