Immersive Linear Algebra – textbook with fully interactive figures (2015) | Hacker News Reader

Immersive Linear Algebra – textbook with fully interactive figures (2015)

immersivemath.com

1,015 pointssebg74 comments

Last year, i saw the videos by 3Blue1Brown and inspired by it went on to read some of the standard text books like linear algebra application by Strang. I had seen the Strang videos earlier but somehow did not follow it through. This time however my perspective was changed. I was approaching the subject through the lens of intuition and simplicity. Wherever i found something challenging, i waited for the next day to again re-do it (because i know that this should not be that complex to understand or the understanding is wrong). And to my surprise, again and again, the difficulty was in my rigidness in understanding. The next day , or even later during the day, when my mind was fresh again, i can reason through the concept and get the intuition behind it.

Since then i have seen the Strang videos again and again. Beginning to end. Read the book chapter by chapter and exercises by exercise. And what a delight it had been. And then i jumped upon Joe Blitzstein's probability lectures. What a blast ! Is there a list of teachers like these there, who in the pretext of teaching algebra/probability etc are in reality wiring up our thinking process in ways immaterial to subject they are teaching. Many of us don't want the material to be too casual/layman terms (which hampers self understanding as its no challenging anything within us) and not too rigid (where we cannot break through the challenge).

> the difficulty was in my rigidness in understanding

At what point did you realize this? Like, could you provide a specific example of a topic you thought was hard at first but later came back to and realized was all about the intuition?

Not the OP, but...

I was just reading Landau & Lifshitz' "Statistical Physics", and can reflect on a series of thoughts that may elaborate on how intuition plays a role in the enjoyment and understanding of complex material. I've been meaning to write it down anyway...

On page 3, the book says "A fundamental feature of this [closed system/open subsystem] approach is the fact that, because of the extreme complexity of the external interactions with the other parts of the system, during a sufficiently long time the subsystem considered will be many times in every possible state." When I first read this, I thought "non sequitur, but whatever, I'll continue..." Now, the context of this quote is that the authors are trying to explain why statistical methods work at all. And they said prior that we start with laws that apply to 'microscopic' particles and use statistics to generalize to 'macroscopic' systems.

However, the second time I read this, I kept thinking it must be backwards. We didn't understand the motion of (classical) protons before understanding the motion of macroscopic balls. So we had to have been operating under the assumption that the macroscopic laws must apply to microscopic objects, and then require that the must also be reproducible macroscopically through statistical methods. That is, we require that these laws be invariant across the microscopic-to-macroscopic transition. But to do that, we have to use a framework which expresses such a transition. So, for instance, if we are reasoning about the motion of a ball, we have to translate our laws into laws over the motion of some statistical model of the ball. Say, it's center of mass. And with this concept in hand, we could write laws that apply to both the macro and micro worlds, since a 'center of mass' is a macro-micro-scale-invariant abstraction. So we partition the space of all possible laws of nature, and chose to work only in that partition which encodes things we can actually know about nature -- the partition identified by the macro-micro-scale-invariant.

So re-reading that passage, it is now not a non-sequitur for me. Now it says "because the interactions with the outside world are so complex, we could not hope to predict their influence. Thus we are justified in using random variables to model their influence, and concepts that derive from the use of random variables to ensure we have complexity-scale invariance when we formulate our laws of physics." And this is not a non-sequitur to me. It follows directly from the meaning of the word "random." Of course statistical methods work when the complexity of a system is indistinguishable from randomness.

And this whole line of thought generalizes (albeit informally.) For instance, the meaning of words is an invariant across a long thread of contextual translations, and these invariants are used the same way: to partition the space of all possible meanings in such a way that one partition contains all of the 'knowledge' imbued by that word, and thus you can navigate a narrower space of meaning to find the intended and/or correct one. Gives me a certain brand of appreciation for good poetry.

Or -- my girlfriend -- who recently told me that she loved algebra but couldn't understand trigonometry. I tried telling her that the algebraic transformations were invariant-preserving operations selected because they conformed to known laws about 'functions' like addition and multiplication which have commutativity and identity laws, and that trigonometry was no different: different functions, but all of the algebraic transformations you needed were selected from the laws of trigonometry with the purpose of maintaining the exact same invariants. (Not that she cared much, to be honest...)

And on and on... I could probably talk for days about all the different ways every subject can be reduced to transformations and invariants and how they are used to solve problems.

Insightful comment thanks for your anecdote.

Thanks for writing that.

> I could probably talk for days about all the different ways every subject can be reduced to transformations and invariants and how they are used to solve problems.

I find myself partial to this type of world view too. I believe it is part of the appeal of functional programming, at the basest level, to shape the programming model into transformations (functions) and invariants (state).

This time however my perspective was changed. I was approaching the subject through the lens of intuition and simplicity...And to my surprise, again and again, the difficulty was in my rigidness in understanding.

Thank you for penning those words. I hope people realize the significance of what you just said.

Perspective is key. You could say it is the KEY -- the key insight into unlocking everything else. I had a similar experience in 2009, and once you have the epiphany -- once you experience the awe of a shift and recognize the implications -- it's like your mind becomes unshackled. You realize you have been blind, and you've just learned to see. And in that flash you gain a deep, visceral understanding of what Alan Kay means when he says, "A change in perspective is worth 80 IQ points," and "We can't learn to see until we admit we are blind."

"Life's Illuminating Perspective" (2009) http://jamesthornton.com/manifesto

Could you link to the probability lectures you're referring to?

Presumably...

Statistics 110: Probability - Joe Blitzstein, Harvard University http://projects.iq.harvard.edu/stat110/youtube

Thanks. Yes, that's the link. Joe is one of those teachers, who try to simplify the concepts while preserving the elegance of what is described. Kind of like minimalistic simplicity in teaching. While teaching, every time he would emphasise the intuition behind them. He avoids the complex formula's/algebra, calling them ugly ... and often presents one line proofs to them !!. After getting through the first few of lectures, you seem to get the underlying trend in his approach. That all the complexity is just hand waving over the the simplicity of underlying concepts. (Sometimes i think if the maths is complex to explain a phenomenon then we are not using the correct theory to explain it. But i am not a mathematician .... )

If you're interested in more great animations for a visual understanding of linear algebra, I can't recommend 3Blue1Brown's "Essence of Linear Algebra" series on Youtube highly enough.

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

These are the best linear algebra videos I've ever seen. The creator is now working on an introductory calculus series. He's already released the first few episodes to patrons.

Wasn't he hired by Khan Academy recently? It's great if that helps a wider audience to benefit from his work.

I think it's the reverse. He did all or most of their multivariable calc series, and maybe some other things for them.

And then started to do his own thing a little more.

Holy shit. I wish I'd had access to these videos during my Mattix Algebra (I think it was called) course in undergrad. I feel like my lack of truly understanding LA has somewhat impaired my grasp of some of the other topics covered in CS undergrad and grad school.

Seriously... the hallmark of all the linear algebra classes I had was the instructor completely failing to demonstrate the various concepts in visual terms. And that's pretty unforgivable, since linear algebra feels like one of those subjects where, until you have some key components of it modeled in your head that serve to glue everything together, every new may as well be random words pulled from a foreign dictionary.

Some of this comes down to learning-style. It sounds like your lecturer was not a visual thinker. Some people really like to just get the algebra and have-at-it (not me tho.) The "model" you refer to doesn't need to be visual. There are applications of linear algebra where the visual stuff just doesn't help much.

Although the focus with this book is with the fully interactive figures, I'm more impressed that ALL of the book is available for free.

I'm constantly seeing more people coming out with books for free or just a passive donation link. This makes me immensely happy seeing as how they're leveraging the available free resources (Latex, CC-BY-SA content, free software for graphics) to make more resources available for free.

Open software is one thing, but a book is much more permanent in my opinion. A book like this will never go 'stale' or old like software does. We only need handful of good books for every topic out there at which point we can basically not buy books anymore. For many topics, I hardly have to consider buying a book since I can just use a free book offered by a professor. And watch course lectures.

What I want to say is this: Please write more for free. It doesn't matter if there is not much interest in what you are writing. It will help you too!

> Although the focus with this book is with the fully interactive figures, I'm more impressed that ALL of the book is available for free.

That was exactly my first thought. "Wait a minute, this is an entire textbook, with all the work, and the prior experience, hiding in plain site behind cool interactivity."

Well written software doesn't seem to go stale, or at least not quickly and poorly written books do.

Well, now that I think about it, the software going 'stale' depends on the domain.

You can write a networking stack in C adhering to a protocol and be done with it, while code for the new $COOL_WEB_APP might go out of mainstream use by the time you're done writing the networking stack.

I don't think most books, especially math and science ones go stale. A poorly written book is just that; a poorly written book. It's bad from the beginning

This is very nice.

As a suggestion for improvement, consider allowing the learner to edit the formulas which represent the figures and have the figures update. Additionally editing the figures could update the formula in real time.

This sort of bi-directional instant feedback will aid the understanding and engagement of the learner better than figure manipulation alone.

Sounds like a good idea - one of the main ways I learned programming... and still wind up doing all the time is to just copy examples and hack them into shape, understanding what's necessary, what's not and how they'll respond to different parameters via experimentation.

One thing missing in comparison to most other textbooks is a set of problems to test the reader's understanding. The rotating and interactive figures are a very nice touch though.

Along the lines of interactivity, maybe having a scratch area like a Jupyter notebook would be a potentially great addition so that I could try problems near the area where I'm reading.

I thought the same thing. A set of problems to test the reader's understanding and a scratch area like a Jupyter notebook would improve the interactivity.

I've recently worked on http://dspillustrations.com for a pictorial description of signal processing concepts. In the online version, it's not fully interactive, just animations. But after downloading, you can certainly change formulas and run it interactively.

Would be interested how one would manage to include a jupyter scratchpad in the online version?

Embedding Jupyter Widgets in Other Contexts than the Notebook: http://ipywidgets.readthedocs.io/en/latest/embedding.html

Temporary notebook service: https://github.com/jupyter/tmpnb

Jupyter javascript plugin for static sites: https://github.com/oreillymedia/thebe

Also, you may want to consider trinket: https://trinket.io/

http://www.hnwatcher.com/r/984396/Embed-interactive-Python-a...

For example, this book uses trinket for interactive Python: https://books.trinket.io/pfe/index.html

I've just tried running:

    import numpy as np
    from scipy import linalg
    A = np.array([[1,2],[3,4]])
    print(linalg.det(A))

at https://trinket.io/features/python3 and it worked like a charm :)

Thanks for these great references! I will definitely have a look at them and see if I can use them for my purposes.

So glad someone made this. I'm a "true believer" in animations and interactivity for learning math. They can make some concepts instantly intuitive that take a while to grasp symbolically.

Discussed in 2015: https://news.ycombinator.com/item?id=10183725.

I'm glad it's been reposted - writing an ana exam in 4 weeks!

on a tangent, I was just reading https://medium.com/@dominikus/the-end-of-interactive-visuali...

in short (over simplified, my take): interactive visualizations are dead because nobody interacts with them.

wondering if this holds here as well.

Can't find the reference anymore, but there were also papers in educational sciences that interactive books usually don't increase comprehension of kids (they just play with them instead of depen their understanding).

edit: sorry didn't want to sound too critical. The work is awesome (upvote), was just thinking out loud.

For me this gets to a deeper point. While there are probably some people who innately understand tough mathematical or scientific concepts, it's been my experience that learning is largely about spending enough time struggling diligently with the material until you've built a robust and correct mental model of it. Then on to exercises to see if your model breaks or holds up when strained. If you approach interactive aids and videos and images with this mindset, they can be very helpful.

I can also clearly remember times when this was not my approach, and in those times, pictures were just a lot of time saved because I didn't have to read text that otherwise would have been there.

Thank you for the link. I learned from it.

I am the author of a widely-used LA text, and have considered adding interactive stuff. But there is a tradeoff. For one thing, it locks you to online, and despite the claims of our IT people, my correspondents (mostly self-learners) do not want online, they want print or PDF (as do I, since the appearance that LaTeX gives me is important to me).

For another, the tech has in the recent past changed so fast that maintaining the interactives would be a significant job. I don't mind learning JS to do something good but tying myself to many hours a year responding to bug reports from people on obscure platforms, or using IE6, is not a good use of my life energy.

Finally, I had a colleague, a complex analyst, try Visual Complex Analysis and he reported that students did not get it. He is very sober, very caring, very reliable. This starts to make sense of his report.

I'd be interested to know your thoughts on the Essence of Linear Algebra videos by Grant Sanderson. What role do you see material like that playing? (They are fantastic if you're not familiar) http://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xV...

Yes, I admire them. I have some technical quibbles, as a fellow educator, but I admire them a lot.

First, note that Needham’s book Visual Complex Analysis is not interactive. It’s just a book with slightly more pictures and more geometrically motivated explanations than most books have.

Personally I don’t think it’s ideal to use as an only book for a complex analysis course, but I found it very helpful for crystalizing my intuition/thinking about the subject. I would recommend every university math student try reading it, especially after going through a traditional course. (Also recommended is Wegert’s book Visual Complex Functions, but definitely not as a primary text.)

Finally, I suspect switching primary books is only going to work well if the professor makes a commitment to teaching using the same explanations. If the book goes about the subject in a completely different way than the lectures, I can well imagine students working homework sets on a short deadline may get confused.

One possible explanation for that: the visualizations are only useful to those strongly motivated to learn the material itself (as opposed to doing well in a class/test or whatever). So, they may not be effective when casually presented to a group of relatively disinterested students—but they may be the perfect thing for some autodidact browsing HN.

To me this book looks like a whole bunch of equations, with some fancy graphics sprinkled on top. And, far too many equations! Linear algebra is much more elegant (simple) than this. To pick one example, they define the inner product using the cosine of the "smallest angle between the two vectors." Sure if you want to calculate a number (an inner product in two dimensions), and you happened to know the angle in question, this might be helpful. But otherwise it completely obscures everything else about the inner product. How does an interactive graphic help you understand wtf is a cosine doing in this equation? What is a cosine anyway? Where is the graphic for that?

This just seems too backward and over-done to me. But go ahead and test it on some newbies, maybe I'm totally wrong here.

More importantly, the definition of the cosine is given by projecting a unit vector onto a given directed line, and then measuring the length of the projection.

The way to find a given cosine of a given angle between two arbitrary vectors is by taking the dot product of the two vectors and then normalizing by their lengths.

Using the cosine to define the dot product is precisely backwards.

Try use web workers to not block UI while processing: http://immersivemath.com/ila/ch05_gausselim/ch05.html

Does anyone know more about how the figures were developed?

I found https://www.lth.se/fileadmin/lth/genombrottet/LUkonf2015/41_... which says "The figures were programmed using JavaScript using a graphics engine that we have developed for the interactive illustrations", but that's all the detail it gives.

Is there anything in particular that you would like to know?

Great content, makes it much easier to get through concepts. But as most math learning resources, it lacks examples of their practical application. "The law of cosines is a very useful formula to know" - fine but why? Most learners can't imagine where they can apply what their teachers say, which brings it down to learn useless-for-them terms, needed just to pass exams. Some basic examples like these would make it much more reasonable to learn: http://study.com/academy/lesson/solving-real-world-problems-...

You might be interested by this curated list of explorable explanations [1]. It has a good section on math.

[1] https://github.com/sp4ke/awesome-explorables

If you like this, you might be interested in /r/explorables, a subreddit filled with interactive stuff:

https://www.reddit.com/r/explorables/

I feel as if I'm looking at the future of textbooks. This is absolutely incredible.

Mandelbrot cites his ability to "think in pictures" as fundamental to his process and insights.

He offers some interesting reflections and anecdotes on this subject in this interview describing his classes préparatoires aux grandes écoles. Apparently his teacher considered him a total wildcard case who would either flunk the exams or pass with flying colours, because of his habit of approaching everything through geometric intuition rather than symbolic manipulation.

https://www.webofstories.com/play/benoit.mandelbrot/6

I'm a MSc in CS student at the university where the authors (whom I've met a few times) teach.

If you have any questions you think I'd know the answer to, ask away!

Perhaps you can let them know their work is being discussed on HN :)

A good idea! Although now we are already aware. If you have any questions you can ask us here or at immersivemath@gmail.com

The order of the words and suffices is like in Turkish. One of my English teachers who has lived in Japan for a time (and she was a Turk) told us that it is very easy to learn Japanese because it follows the same word orders, and vica verse. On the other hand English is very hard becuese there is zero connection.

Amazing resource, but is it still being updated? Last update seems to be from back in July last year.

New chapter out today! This time on linear mappings.

Excellent work! I've been trying to get my mother (she's a physics teacher) to learn linear algebra properly for a long time. Artin didn't work (ha), Khan Academy moved too slow/bored her, but she seems interested in this.

It's important to appreciate how useful it might be to make math "tangible". Sure, someone who can define a manifold by saying "oh, put charts on it, locally diffeo blah blah" probably has a good set of mental models that help them find analogies and even "tangibilize" (word?) new ideas. Once you learn the way abstraction works, broadly speaking, you can take the training wheels off: but lots of people never get past that stage. On one hand, I see a lot of people on HN talk about how the complicated notation of academic math/CS keep people out (and there is an understandable amount of resentment at people keeping "outsiders" out with this), and on the other hand, I sort of reflexively bristle (it's gotten lesser now) at people integrating the notion of an inner product into a vector space, because it is important to not stumble later when you find out your basic intuition for something is broken[3]. (Of course, intuition can be incrementally "patched": Terence Tao's essay[3] talks about this, from the perspective of someone who is a brilliant educator in addition to also being one of the most versatile mathematicians around.)

Maybe presentations of basic mathematics that are

- simple

- rigorous

- free of half-truths

can be made accessible by using such visualizations and interactive techniques to decrease the perceived unfamiliarity of the ideas? I don't think there are many[2] treatments of mathematical topics that satisfy these criteria and yet manage to be approachable: one either skimps on a clean presentation (Khan Academy), or assumes a lot of mathematical maturity (shoutout to Aluffi!) from the reader. "Manipulable resources" might help fill this gap. It's an exciting time!

In the section where they give examples of matrix inverses, to give people a sense of how important multiplication order is, they give an example of RHR'H' (using a prime for inverse, R for a rotation matrix, and H for a shear matrix). One of the most beautiful illustrations[1] in the book follows, with the four corners of a square moving independently in circles, and then the book states that

    It is quite close, but it is not at all useful.

While I understand the need to clarify the importance of multiplying matrices in the correct order, maybe a short aside on the unreasonable (practical!) effectiveness of commutators[0] would be useful?

[0]: https://en.wikipedia.org/wiki/Commutator

[1]: http://immersivemath.com/ila/ch06_matrices/ch06.html ("Example 6.12: Matrix Product Inverse au Faux")

[2]: Visual Complex Analysis is brilliant, though.

[3]: https://terrytao.wordpress.com/career-advice/there%E2%80%99s...

You have to show her the videos from 3Blue1Brown. They're so good

Visual Group Theory by Nathan Carter is another one I have been enjoying.

This is fantastic, and where I have expected e-textbooks to go for years. Kudos!!!

Reminded me of Explorable Explanations: http://explorableexplanations.com

Q: How would one go about creating such animations? What's the best workflow? Found the figures were slow to load.

Great resource, though!

I was just wondering recently why something like this didn't exist when trying to learn about Matrixes via Kahn Academy.

This looks great! Could some of you share other nice books and video series covering core CS math basics?

I really like this. This would've been handy as a student. I definitely prefer this to a textbook.

Jaw dropped. This is exactly what I needed to get some progress with my shader learning :)

You might also like https://thebookofshaders.com

I wish we had resources like this when I was on the university.

Wow...i so so so much wanted a book just like this... Thanks

This will come in handy for my LinAlg exam :3

Can anyone hack this and make a PDF book?

Doesn't that defeat the point of intractability?

Interactivity maybe, intractability is not having the property of being tractable. For an algorithm, this means that its performance can not be described in polynomials. But your point is still valid, a pdf does not support animations, thus a pdf would be useless.

I didn't ask PDF for interactivity, i asked PDF for taking notes. The theory is so solid.

Wow, this is great.

Al right, where do we get it?

Reply on news.ycombinator.com