Matrix Multiplication | Hacker News Reader

Matrix Multiplication

matrixmultiplication.xyz

873 pointskasbah131 comments

This is a cool example of what Bret Victor calls an "Explorable Explanation" [0]. That said, I feel that it's more important to understand how and why matrix multiplication corresponds to a composition of linear transformations than learning the actual mechanics of doing the computation. You can get good at matrix multiplication without knowing what is going on. I view that as a less valuable activity than learning about linear transformations (vector-space structure preserving mappings) between finite dimensional vector spaces.

[0] http://worrydream.com/ExplorableExplanations/

Bingo. We did matrices in high school and even through a first linear algebra course in undergrad I got more or less competent at the mechanics of matrix multiplication without ever having any sense of why anyone would want to do this or why these rules, as opposed to some other set of rules that were equally formally valid, were of any interest. I could even tell you what the eigenvectors of a matrix were but had no sense of what that actually meant in terms of a linear transformation that matrix represented. Linear algebra is probably the area of math that is both universally important and completely skimped on by our education system.

I feel very similar. Do you have any recommendations for resources to learn about why matrix multiplication is defined that way?

Check out this series of videos "The Essence of Linear Algebra"[1] for a really powerful visual and intuitive explanation. It starts with vectors and builds to matrix multiplication and further to several other topics.

[1]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

"All the mathematics you missed but need to know for graduate school"[1] helped me a lot (and, in fact, I had and did).

Once I had finished that, Cullen's "Matrices and linear transformations"[2] was really helpful too. But I wouldn't do Cullen if you're still, as I was, floundering with the concepts of why you're doing this in the first place. It's great once you have those concepts down.

[1]: https://www.amazon.com/All-Mathematics-You-Missed-Graduate/d...

[2]: http://store.doverpublications.com/0486663280.html

Garrity's book looks exactly what I've been looking for for brushing up on basic math skills after a decade of fairly low-brow work after my MSc. Thanks for the reference!

Hope it helps! It definitely got me through the first year of grad school...

Let's say you have a system with N possible states, evolving in discrete steps. At each step, the system has some probability of switching from any state to any other state. That gives you an NxN matrix of switching probabilities. For example, if the system always stays in the same state as it started, the switching probabilities are an identity matrix (1 on the diagonal and 0 everywhere else).

Now let's see what happens after two steps. If the system started out in state i, what's the probability that after two steps it will end up in state k? Well, it's the sum over all possible paths. In other words, the sum of probabilities of i->j->k for all possible j. In other words, the sum of p_{ij} times p_{jk} for j from 1 to N. But that's exactly the definition of multiplying a matrix by itself.

Now it should be easy to understand that whenever you have matrices that represent transformations of some object, composing transformations will correspond to multiplying matrices.

If you compose two linear transformations into one step, and apply distributive property, you get the matrix multiplication rule.

http://math.stackexchange.com/questions/31725/intuition-behi...

Axler's book, "Linear Algebra Done Right", is widely considered to be one of the best texts on linear algebra that focuses less on the mechanics and more on the structure of linear operators on vector spaces.

http://linear.axler.net/

Yes, and the abridged version (no proofs, examples, and exercises) is downloadable for free. He avoids determinants altogether (well, until the 10th chapter).

http://linear.axler.net/LinearAbridged.html

In particular understanding that conjugation by another matrix is just a coordinate/frame transformation is crucial.

Relying to much on explicit coordinates many times obfuscates what's going on.

Is that true for non-square matrices?

Conjugation only makes sense for square matrices. More generally, left and right multiplication correspond to coordinate transformations of the output and input space, respectively.

If A is n x n and P is n x p then P is not square but P^T A P is well-defined.

For understanding the mechanics of matrix multiplication, I found it useful to think in an analogy consisting of a grid of two layers of pipes; One -- the input-pipes -- coming in one direction, and the other -- the output pipes -- laid in an orthogonal direction. Then there would be "taps" in the cross-sections between the input and output pipes, representing the numbers (multiplication factors, really) in the matrix. I illustrated this in this little drawing: http://imgur.com/gallery/gBs64

The point then is that the taps (again, representing the matrix values) determine how much of each item in the input vector, that should be mixed into each item in the output vector.

This analogy has the limitation that the taps are allowed to enhance the flow, not just limit it, like physical taps would. That is, outputting more than 100% of the input :P Also, while this way of illustrating it may make some sense for matrix * vector multiplication, matrix * matrix would probably become a prohibitively cluttered image.

I really like that. And matrix * matrix would just have to be 3D, with a whole cube of taps. Sure, not easy to illustrate, but the concept extends well. It also makes it more obvious what the complexity of the operation is.

In a little more detail:

Look at the animation in the article, after the second matrix has been rotated and put on top of the first one. Then flip its top up so the two matrices are orthogonal. Finally, rotate the whole thing 90 degrees to the left (rotating along the axis that goes from the top of the page to the bottom of the page). Now you can see that the result will be a 2x3 matrix. And the values in each spot will be the sum of the products beneath it.

Actually the complexity of matrix multiplication is not known. Current best is n^2.37

There is a proven lower bound [1], though.

[1] http://epubs.siam.org/doi/abs/10.1137/0218052

That's pretty cool. For me, everything changed when I started thinking in terms of column spaces and linear transformations. Matrices are not just a series of multiplications and additions. They describe so much more.

3Blue1Brown is an amazing youtube channel that provides very entertaining and educating visual explanation for all sorts of maths.

He has a whole series on linear algebra, including matrix mulitplication:

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

... and after a succesfull Patreon campaign[0], he's going to do "Essence of Calculus" next. I'm seriously excited!

[0] https://www.patreon.com/3blue1brown

I discovered this last night! I've been dealing with matrices frequently for the past 15 years and never really got them. Having watched his series now, I feel like I completely understand them, in a way that I never even fathomed before. I think I'm going to make a small series producing a raytracer from first principles, in a somewhat similar style; that visual design was just the perfect medium.

Thanks, I had not seen that channel before.

This is why I'm a big fan of Axler's Linear Algebra Done Right. The book's emphasis is on the concepts behind the calculations rather than the calculations.

I actually disagree with Axler on his avoidance of the determinant, though. I wish instead of avoiding it he'd spent more time developing it conceptually, as it's actually a fascinating construction. But to this day I have yet to find a gentler and better introduction to serious linear algebra than his book.

He recommends it as a second course, but I read it during my first course in the subject and considered it my "secret weapon". I truly believe that book is what allowed me to get a perfect score in the class -- I had a conceptual understanding that was just not possible to glean from the official course textbook.

Axler's book is very good. An alternative to consider is Paul Halmos's much older Finite-Dimensional Vector Spaces [0]. Both books take the same basic approach, and the proofs of the major theorems are substantially the same. (Halmos's book is well known and well liked and was probably Axler's starting point.)

Axler's book covers more ground (most notably, Halmos presents the polar decomposition but not the singular-value decompostion) and uses more modern terminology and notation. But Halmos's book has the merits of being half as long and a third as expensive, as well as having been written specifically to prepare the reader as directly as possible for Halmos's short introduction to Hilbert spaces [1].

I highly recommend one or the other of these books for readers who want to understand linear algebra as mathematicians do.

0. https://www.amazon.com/Finite-Dimensional-Vector-Spaces-Paul...

1. https://www.amazon.com/Introduction-Hilbert-Theory-Spectral-...

Axler's book was free for me from my university's SpringerLink thingy.

Due to your post, I checked mine -- free pdf version available! Bingo! Thanks for the tip.

For those who already know enough linear algebra to be bored with an entire book dedicated to Axler's approach to linear algebra, you can simply read the paper upon which the book is based:

http://www.axler.net/DwD.html

I can see why he discourages it. It prevents understanding why a matrix is singular. I often find students who use it as the main justification for singularity without further understanding singularity -- namely that it occurs when a linear map sends things to a lower dimensional space than it accepts.

This is why I'm a big fan of Axler's Linear Algebra Done Right. The book's emphasis is on the concepts behind the calculations rather than the calculations.

I could never learn mathematics that way, and goodness knows, school systems in both Europe and America have tried to teach it to me that way. What I understand of mathematics today has mostly been achieved through autodidaction.

Some of us cannot, repeat can not go from the abstract to the empirical; I'm one of those people, who have to go from the empirical to the abstract instead. I need to see the mechanics of it, because I learn by observing the pattern. If I cannot make out the pattern, I cannot comprehend the abstraction.

Take UNIX manual pages for example: the first thing I go to after reading the SYNOPSIS is the EXAMPLES section (and since most GNU/Linux manual pages have no EXAMPLES sections, it's a product which is useless to me, unlike UNIX). The more examples the EXAMPLES section entails, the faster I'm able to grasp the concept. Same with http://matrixmultiplication.xyz/, if I could have learned matrix multiplication that way, visually, instead of having to do it in my head, it would have been far easier. The other way around, it was a slow, painful torture, and to this present day, I cannot multiply the matrices in my head, or without these visualisations to remind me of the rules: they're too complicated for me to keep in my head.

The point I'm trying to make is that not everyone learns the same way, and our brains do not process the information in the same way, even if the final result, understanding of the concept, is identical. A lot of abstract concepts which are difficult to reliably repeat, or whose outcome is either non-deterministic or unclear, we do not even understand the same way, hence opinions often differ by a small or large margin. Don't assume that the learning technique which works for you would work for someone else. That's a major failure of most pedagogical approaches, with the exception of perhaps Montessori and Fröbel. Unlike Montessori and Fröbel, most pedagogical approaches force a uniform way of learning, except that a human brain does not learn in a uniform manner, differing from individual to individual instead. Teaching should be a highly individualized approach.

But surely these sort of meaningless examples with randomized numbers like at the site aren't that useful? I mean, a "real-world" example of matrix multiplication would be having a proper translation or rotation matrix and then visualizing how it maps a point to its image.

That demonstrates multiplication of a matrix with a vector which is being rotated, but multiplication of two matrices corresponds to composition of the corresponding operators.

This is actually pretty instructive. For instance, rotations in three-dimensional space do not commute in general; hence neither does matrix multiplication.

Technically matrix-vector multiplication is just a special case of matrix-matrix-multiplication, but I take your point. Indeed it would be illuminating to see how exactly matrix multiplication composes two transformations.

Is there something like this regarding homogeneous coordinates and the projective plane?

It's strange. Somehow I have now problems whatsoever regarding the use of 4-component vectors. As long as the z component stays one, I'm all set. But lately I'm trying to understand the construction of projection matrices and how to apply the knowledge of homogeneous coordinates, for example shadow mapping.

Parallel lines suddenly intersect? 3D space is just a projective plane? Now everything is all weird...

Homogenous coordinates have accrued really overcomplicated language around the very simple idea that something appears half as large twice as far away.

Not sure if you already are aware, but you didn't use the term, so I figured it might help to point out that what you are talking about is projective geometry. I'm not sure of any good texts, though it sounds like you might want one more focused on graphics than projective geometry purely in the abstract.

It at least cuts down on the amount of time you have to explain the mechanics of it. This is more intuitive to memorize than an algorithm composed of bullet-point instructions. For intuition, I highly recommend https://www.youtube.com/watch?v=kjBOesZCoqc - I think both the intuition and practicing the details should be learned.

You'd probably like this then:

https://graphicallinearalgebra.net

This is cool, but I have a few criticisms:

- Flipping something over like that looks cool, but confuses the brain. At least it confused mine.

- Inconsistently calculating 1 or 2 elements at a time is confusing. Having 2 results calculated at a time simply put requires the recipient to untangle what is going on. This only gets worse when the recipient tries to do it themselves for examples with more columns in the right matrix.

Disclaimer: I am biased as I already know how to do matrix operations, and I visualize it differently. When multiplying by hand I translate the right matrix up, like in the animation, but do not flip it over. Then I calculate each element one at a time by taking the sum of products of the row and column that intersects over that element. Like so: http://i.imgur.com/D572zOI.png

As you can see I'm not much of a graphics artist.

Edit: actually, I think this would be really good if you "filter down" one column at a time, instead of moving the whole matrix down, so that it only calculates one element at a time, one column at a time. Means the animation will contain more steps, but I think that is actually to its benefit. Does this make sense?

> Flipping something over like that looks cool, but confuses the brain. At least it confused mine.

I think it would be helpful to have a shadow of the right matrix after it's moved to the top. This way it would be easier to see where particular numbers are coming from in the original input.

Disclaimer: I'm similarly biased and actually the reason I came up with this shadow thing is because I also imagine the right matrix moving like here but somehow this never causes it to disappear from its original place on paper, go figure, so this animation looked really alien to me.

Not sure if anyone here has mentioned this yet, but it's useful to think of matrix multiplication as the most natural extension of one-dimensional algebra into multiple dimensions. First, remember that people first wrestled with and found solutions to "1-D" equations like "ax^2 + bx + c = 0". When others came along asking about how to extend these results into multiple dimensions so that previous results might still be useful, the most natural answer was found to be how we now define matrix multiplication. To see further evidence of this, take your favorite linear algebraic equation using matrices and then look at the edge case where you have only 1x1 matrices in your equation. Usually you'll end up with something that looks analogous to a result in basic algebra in real number (or complex) variables, not matrices.

If all this still seems counter-intuitive, then realize this is essentially why math is so important to our logic. It helps us to reason about things that are not obvious, yet logically must fall through from the basic tautologies we "chose" (like 0 != 1, addition on integer numbers, etc.). Or, more aptly in this case, it helps retain an analogous symbolic representation (the overall logic of the equation at the high level) despite a change in the underlying nuts and bolts of computing solutions to it (matrix multiplication vs. our regular "1D" multiplication).

I'm sorry, but that's not how that works.

Your example is of a second-degree equation, which can't be solved by linear algebra. Linear algebra was invented to solve systems of linear equations, and matrices first arose as a convenient notation for expressing those. Matrix multiplication came later and is now understood to correspond to the composition of linear morphisms on vector spaces.

If you try to extend the (very few) methods we have for obtaining exact solutions to polynomial equations in one variable to higher dimension, you (a) fail and (b) invent algebraic geometry.

I like the idea, but my brain is wired differently.

Usually, when I do matrix multiplications, the rows on the left side transition to the columns on the right side. In the example, the 1 goes to 2, 2 to 6, 1 to 1.

The animation completely confused me, because it looks like being reversed, but the result is the same.

Maybe, make this a visualization option?

Essentially, that is how I was taught and do matrix multiplication as well. But this sliding window approach is also interesting. Although, it would be a bit complex to understand how to fill in the results based on the sliding window.

I do this too and this is way more intuitive in my opinion, as you can easily see which entries of the resulting matrix are linear in which rows and columns. Also, you don't need to transform the matrices in a way that is impossible to do on paper.

Me too. This is like seeing someone write legible English starting from the right hand side of the paper.

Not wrong, but discordant with my expectations.

There are several other visualizations that can be useful too:

  Multiplication
  We can see matrix by matrix multiplication from 5 different positions:

  row by column multiplication
  column at a time
  row at a time
  as sum of outer products
  block multiplication

  All of them are equivalent and lead to the same result

http://mlwiki.org/index.php/Matrix-Matrix_Multiplication

Does your mental model fit into one of these, or is it also reversed in the order it is applied? It would indeed be interesting to allow visualizations of all the approaches.

For me personally, a simple animated version of this image[0] looping through rows of A and columns of B seems easier to understand.

[0] https://upload.wikimedia.org/wikipedia/commons/e/eb/Matrix_m...

The idea is good but I am afraid it misses the point entirely. Multiplying both columns of the right hand side matrix to the left hand side matrix is really not helping the target audience understand what is going on. 2 operations are going on as the same time. I believe https://www.youtube.com/watch?v=XkY2DOUCWMU is a much better way to illustrate the matrix multiplication (and provide a clear explanation of the why) .

Beat me to it :) amazing channel!

Comments like these (aside from providing a nice link) just make authors of content sad. Why say such thing? You can say "here is another helpful link", but "misses the point entirely"? The point is to provide a self-evident mnemonic, as explained here: https://github.com/staltz/matrixmultiplication.xyz

Not nice.

Mr Staltz, please allow me to elaborate. I did not mean to be mean at all, nor to reduce the value of your work, excuse me if I made you feel like that. I am actually grateful that you made this animation as it allows one to reflect on the concept.

If your kid proudly brings home a drawing, you are most probably going to greet him with warm encouragements no matter the quality of the art. The kid is in a process of developing his creative sense, learning, and forming his/her sense of reality. Comparing the drawing to what mature artists produce would certainly have a negative effect on the kid. The point is to provide a feedback relevant to the recipient of the critic.

The same goes here with this animation. I respect your work but obviously disagree with the usefulness of it. You made an animation to explain the matrix product, which is a really good idea. Alas, I think the realization of it gives a confused picture of what it is. I doubt that a student with no prior understanding of matrix products will get much enlightment. Too many operations are going on at the same time. This means that it misses its intended audience: students in need of building an intuitive understanding of the matrix product. That's what all those javascript (among others) animations (ex: http://jackschaedler.github.io/circles-sines-signals/dft_int...) are all about: make it simpler to grasp a concept. Unfortunately, though beautiful, the animation is not clear. At best it provides a visual mnemonic for multiplying matrices which maybe useful for some but not for the younger version of me in need of understanding :)

I realize that instead of providing such a harsh critic, I would have been better inspired to suggest improvements. Mea culpa :)

I think your elaboration just made it worse, as you compared my work to a kid's drawing, and assumed that the target audience will not benefit from that. I actually have proof against that. In Twitter, over 2000 people have retweeted, and countless people have commented along the lines of "I wish this was around when I was studying in school", or "now I finally understand it", and even school teachers saying thanks and how they'll use it in class next week. And, even this hackernews thread, for that matter.

To back up the parent, unfortunately, the audience is not always qualified to judge the quality of your work - for the simple reason that they have no idea what they are missing!

What they are missing from your explanation is "why". Your animation shows how to multiply two matrices, and anyone confused about the mechanics of doing that might find it helpful.

But the actual mechanics are not of much use if you don't know why they are this way. Everything in mathematics is made up by humans, so why would someone at some point write down this particular set of rules to apply to rectangular arrays of numbers?

To that end, I reaffirm the parent's opinion that your visualization only helps to confuse people, giving them a false sense of understanding. There is already too much emphasis in education on "how", and too little on "why"; the comments from the teachers you mentioned are downright scary to me.

Perhaps you will take this comment as yet another critique that is too harsh - but it must be done, so that others don't mistake your animation for an explanation of concepts behind the operation. When one understands the concepts, visual mnemonics - like the one you created - are not needed.

To start a productive discussion, however, (which, hopefully, might lead to you using your skills to come up with a version 2 of your work), I am very curious to hear your answers to these questions - which people confused about matrices might have:

1. What information is actually stored in a matrix? Why write down a bunch of numbers in a rectangle instead of, say, a list or a triangle (like Pascal's triangle)?

2. Why would one want to "multiply" two matrices? That is, why would one want to combine two matrices, and what information would the "product" matrix store?

3. Why do the rules for "multiplication" actually yield the result desired in (2)? Wouldn't some simpler rule do the same? For instance, if we have two 2x2 matrices, why not just multiply the numbers there elementwise (top left by top left, top right by top right, etc.)?

My claim is that anyone who can answer these questions wouldn't need your visualization - and, conversely, your visualization doesn't help answer these questions at all.

I'd be glad to be proven wrong, so I'm looking forward to your explanations of the three questions above.

I totally acknowledge that it is missing the "why", I knew that from the beginning. The widget I built simply answers the "how". It's a very focused tool, and this kind of tool may be very useful in itself. It's a tool to be used in the context of other tools, like books and courses. Your ordinary calculator never answers the why question. And if the audience, who includes both students and teachers, isn't qualified to judge the quality of the work, who exactly is?

On a side note, it's disgusting (I find that an appropriate word) how hackernews threads are so harsh and negative. You can do a fun exploration: search for historical threads announcing tools and services that were revolutionary (Angular.js or React.js, to name a few), and you'll find a plethora of negative comments. If hackernews discussions are good judges of quality, absolutely everything in this world sucks.

> And if the audience, who includes both students and teachers, isn't qualified to judge the quality of the work, who exactly is?

Again, it is very unfortunate, but on the average, high school teachers wouldn't be.

For starters, linear algebra (where multiplication belongs) simply isn't taught at high-school level. It is not in the standard math curriculum in any state. Bits and pieces of linear algebra, like teaching matrix multiplication alone, simply don't belong in the high school curriculum.

For that matter, too many high school instructors either don't have deep enough knowledge of linear algebra themselves, or if they do, they don't have enough experience or expertise in teaching this subject - simply because it is not a part of the curriculum.

To compound the problem, too often we don't teach math educators well enough. The math requirements for math teachers are often significantly lower than for math majors, and many don't get to see too many facets of mathematics.

To answer your question: matrix multiplication belongs in a university-level linear algebra course. Our system is so perverse that the version taught to non-math majors is missing most of the explanations, and even the math majors don't really learn the linear algebra until their second (or even third) year.

So here's my opinion:

The instructors qualified to judge linear algebra instructional materials would be people involved in teaching linear algebra courses at a college level - the professors, TA's, and people running help sessions in universities.

The students would be qualified to judge in retrospect, once they have themselves learned the material beyond the definition.

>On a side note, it's disgusting [...] how hackernews threads are so harsh and negative. [...] If hackernews discussions are good judges of quality [...] this world sucks.

Don't take HN threads as judges of quality - you have seen yourself that they are not. Yes, they are endless sources of critique, and some of the critique is valid and useful. Things people here talk about may succeed in spite of the flaws, but it doesn't mean they are flawless.

And it's great! This thread alone can give you many ideas on where to go from here if you like making educational props like this one - and how to improve what you have done. Treat this thread as a feature request, as a todo-list, as a bug report. You have just got yourself a small army of through beta-testers, who were kind enough to submit detailed reports (with links and examples!). Leverage that.

If you need praise: you are a talented programmer, who can create great visualizations that can help many people get a deeper understanding of things that you have mastered. And the critique is here because of this potential - because you really, really, really can do better.

>I totally acknowledge that it is missing the "why", I knew that from the beginning. The widget I built simply answers the "how".

"How" what?

How humans multiply matrices? Certainly not, I don't cut out one matrix and put it on top of the other. And, to be frank, if I need to multiply two matrices, I either use a computer algebra system (or code it up).

How computers multiply matrices? Software like MATLAB or BLAS would use more complicated algorithms which perform faster (Karatsuba, Strassen, whatever).

In the end, your tool is just a visual mnemonic for a definition. It's a SOHCAHTOA for matrices. And really do think we don't need more of that in education.

> Your ordinary calculator never answers the why question.

That's why there should be way more caution in using calculators in classrooms than is currently exercised. The current way of things is horrendous, but doesn't have to be. I have designed Calculus labs with MATLAB in which, I hope, the computer is not doing anything that the students could do by hand (given a lot of time). Done this way, calculators and computers become tools which aid understanding.

Anyway, regardless of what the motivation of others here was, here is where I'm coming from:

I've taught linear algebra help sessions where by the middle of the semester the students could multiply, row-reduce, invert matrices without having a clue what they are doing. They could put a bunch of vectors in a matrix and do - what to them was - some number magic to check for linear dependence.

I would draw them two arrows on the board pointing in different directions, and ask them a simple question - are these vectors linearly independent or not? They would be thoroughly lost.

That's the state of linear algebra education in the US. We are teaching people mechanical operations that computers do better.

There are better approaches! Linear Algebra Done Right (S. Axler), for example, is one of the few books that justifies the definition of matrix multiplication. Practical Linear Algebra does everything visually, and is suitable for beginners. But the first text is only taught to math majors, and the second one is not widely used.

In any case, I am highly opposed to teaching the mechanics an operation, because with a good understanding, people will come up with the mechanics themselves.

You are an example of this. Having an understanding of the operation allowed you to come up with a visualization. But the visualization, in my opinion, doesn't set one on a path to develop a similar understanding. And this is where your work can be improved.

For instance, take this: https://betterexplained.com/articles/matrix-multiplication/ https://betterexplained.com/articles/linear-algebra-guide/

Is this a perfect explanation? Maybe not, and it's not animated. But the author tries to pass on the way to get a better understanding, a new angle on how to understand the operations being done.

Would doing things like that with an animation be challenging? Certainly. But don't blame the audience for having a high expectation of what you can do.

I don't know how you think that school systems, especially in the west, place more emphasis on "how" than "why". Western schools most definitely place their emphasis on "understanding" rather than "doing". I think that all this does it create a false sense of understanding. Students think they understand why something is done, then when they go to do an actual problem they are totally lost.

I think this is a big reason why american students rate themselves as being very confident in their understanding, but performing rather poorly compared to other nations. I'll have to look for the citation to back that up.

Just my 2cents.

edit: http://www.newgeography.com/content/003840-high-confidence-n...

there were other more recent surveys on this topic, this is all i can find right now

>I don't know how you think that school systems, especially in the west, place more emphasis on "how" than "why".

I've taught people at college level, and have seen way too many cases of people knowing how to perform the steps to, say, solve an equation, find a derivative or take an integral without understanding what the operations actually mean, or why bother with all this stuff in the first place.

It's not the students' fault; the concerns I am trying to voice have been articulated by many over the recent years, and are best summed up here: https://www.maa.org/external_archive/devlin/LockhartsLament....

> Not nice.

100% agree. It reminds me of the "yes, and" rather than "yes, but" thing from improv and stuff like that.

If what you've done helps a single person (and it did me), then it is valid. It might not be perfect, and it might not be right for everyone, but it was useful and worth putting the effort into (and this was. Thank you for it).

As part of the target audience (I didn't previously understand matrix multiplication, but now I do), you are definitely wrong. This does help the target audience understand what is going on.

I think your core fallacy is that all people need to be taught in the same way, and that there is one correct way to teach people. The videos you link to may work for some people, but not for others. This explanation may work for some people, but not others. It turns out not everything works for all people.

>I didn't previously understand matrix multiplication, but now I do.

This belief is why so many people are complaining about the site. Put simply I (and I assume others) do not believe that you understand matrix multiplication as a result of this site.

We could be wrong. Please explain matrix multiplication.

I agree with you that a person that uses this website does not really 'understand' matrix multiplication in any deep way, though they will have slightly higher chance of understanding the algorithm at face-level.

I don't think that makes the site worthless or worthy of the some of the scorn I'm seeing in here.

I strongly agree with the complaint that "this completely misses the point" is super negative and dismissive and should be condemned.

If we-the-community want people to keep making things for fun or to try to help people, we have to criticize them graciously instead of saying that they are so wrong as to be useless. Not because it is moral or proper to do so, but because in practice dismissive and scornful negative feedback makes creators less likely to create new things or to try to improve old ones, and that's bad.

Question for staltz - are you happy with the cycle code at https://github.com/staltz/matrixmultiplication.xyz/ ? It feels like a lot of work for this sort of task which I feel could have been achieved by using an existing (mutable) matrix lib and drawing ad-hoc. It feels like an awful lot of code written - especially with all the `.something === "SOME_STRING_CONSTANT"` (couldn't those be typed directly in TS btw?).

Still, you're a smart guy - I'm trying to see why it's worth it. Do you really feel model-view-intent shines here?

(Nice visualization by the way)

Yes, I'm very happy with that code, apart from one thing: how animations are done sometimes with tweens in JS, and sometimes with CSS.

I'm sure it could have been built with less code. In general, the more shortcuts you take and the less abstraction you use, the less amount of code you will have. But also, it's likely that you would mix concerns more easily. So in general, when separating concerns, you end up getting more LOCs. That's what I've perceived in my experience, not a fact that I can prove.

So I do feel like model-view-intent shines here. Model only has state concerns. I was careful in moving away DOM-related code from the model. So theoretically you could reuse this model for React Native or any other UI target. View only displays state in a visual representation, and intent only captures user events and represents them in a form suitable for the model.

About some TypeScript code, I think you referred to https://github.com/staltz/matrixmultiplication.xyz/blob/mast.... These could have been avoided. The only purpose they serve is to make inference work out for lines like these: https://github.com/staltz/matrixmultiplication.xyz/blob/mast... TypeScript knows that `action` is of type StartMultiplyAction. If we had only done `.filter(ac => ac.type === 'START_MULTIPLY')`, TypeScript couldn't infer anymore. So I understand those isFoo functions are very boilerplatey, but they serve the purpose to provide accurate typings. It's optional and depending how you write TypeScript code, you can use more type casting and more `any`, or you can go down the route of providing accurate types everywhere, which is what I did. This is not at all related to Cycle.js nor MVI.

Feel free to ask more architecture-related questions. I like these. What else feels like "too much code"?

I think a cool way to visualize matrix multiplication would be to put three matrices MxN, NxK, MxK on the sides of a three-dimensional box MxNxK. Each cell of the matrix MxK would cast a "shadow" onto some row in the MxN matrix and some column in the NxK matrix, whose dot product would give the value of that cell.

That's exactly how I finally fixed it in my head permanently.

Would you mind elaborating or providing a link for the uninitiated? I use matrix multiplication in almost everything I do and that sounds like a neat way to visualize what is happening.

It's easier to think of matrix multiplication by computing the matrix elements, which means reducing the problem to NM vector dot products.

To get the element of the result matrix at position (n,m), compute the inner product < r_n | c_m >, where r_n is the nth row vector of the left matrix and c_m is the mth column vector of the right matrix. Once you try it, you'll see that it's also much easier to visualize than this strange, unintuitive approach.

With great power comes great responsibility. That domain name belongs to that person. I hope they do society a favor and put more content regarding MM that that, because they chose one of the least intuitive ways of thinking about it. Put more ways up, or ditch the domain.

If the goal is to get people to understand what matrices are, the best way is to teach them about operators, transformations, vector spaces and linear algebra in general, because this is really the only way to fully understand what's going on without relying on some heuristic.

If the goal is to get people to remember how to do matrix multiplication, at least put up more ways of doing it.

Nope, won't ditch the domain. I'd recommend you ditch that type of speech.

Everyone's a critic, don't let it get to you.

I'm sorry if my words upset you. That said, it would be a shame to tell someone off if they bring up a good point (however brash). I used to teach this stuff, and I can tell you many of my students would end up being more confused with your visualization. I hope you can see that you've assumed some responsibility. That's all I can hope.

Neat visualization. I feel it's important and eye-opening to note that there's not just one way to multiply matrices, there're five (according to Strang):

5 ways to multiply A B = C

1) Each element of C is a dot-product of corresponding rows of A with columns of B

2) Each column of C is a combination of the columns A. Each column of B has the coefficients.

3) Each row of C is a combination of the rows of B. Each row of A has the coefficients.

4) Accumulate outer-product. Accumulate sums of of outer-products of each col of A with each row of B.

5) Block multiplication

They all produce the same answer, but different ones provide different insight for particular situations.

See Video Lecture #3 of Strang's Linear Algebra course for more info: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...

His lectures are awesome, wish some of them could be remastered though

Change to color of rows to red & blue (while multiplying) so it is easier to which result is which product.

This is a great idea

This reminds me of Nicky Case's Explorable Explanation of a 2D transformation matrix: http://ncase.me/matrix/

Awesome.

Some concepts in math are so much easier to grasp with an interactive visual. I once made an entire game around matrix multiplication -- such that players didn't even know they learned how to do this.

And sometimes, visualizations can help cause discoveries. A lot of progress in theoretical physics came about because physicists 'guessed' what nature's equations should look like...and then got shocked when they discovered they were right.

Take James Clerk Maxwell, for instance:

"He had achieved his greatest success by relying on just one empirical fact known to every school boy. Yet now he tried to do without any empirical facts, by pure thinking. He believed in the power of reason to guess the laws according to which God has built the world."

http://www.mathpages.com/home/kmath103/kmath103.htm

I'm curious about this game. Is it online?

I built it for the Windows 8 Store (of all places), years ago, in base C++/Cx. All de-published now though. Could dig up a gameplay video if you wish...

Thanks, but I'm just curious and don't like videos. The world needs more things like that, though -- it was nice to hear of it.

As others have mentioned, many introductions to linear algebra / matrix multiplication only gives the student a formula, but not why the formula looks like it does.

Why is matrix multiplication not pointwise, was something I had a hard time understanding.

Better explained has a good article on this, that really made it click for me

https://betterexplained.com/articles/linear-algebra-guide/

This is nearly how I think of it, but for me I do one column of the second matrix at a time. I think it is clearer that way. This way makes it look like there is some kind of staircase thing going on which there isn't really.

That was my thought too - I was taught to do one row of the first matrix with one column of the second matrix at a time.

I found the whole staircase thing quite confusing.

Matrices are linear maps, and matrix multiplication composes the linear maps. Now statements like the determinant of the product is the product of the determinants, the trace is preserved under similarity transforms (since similarity transforms just rewrite the linear map in a different basis), etc. are intuitive.

I've always felt that these explicit calculations don't really get to the point. You can memorize them and still not really understand what's going on.

When I was a math student, we had two semesters of linear algebra. We only saw matrices at the end of the first semester; other than that it was all about linear mappings.

The thing is, most linear algebra classes that are taught to other sections (biology, CS, …) start directly with matrices and are mostly computational.

Maybe you know something I don't, but I don't find these statements about the determinant and trace intuitive just from knowing that matrix multiplication corresponds to composition of linear maps. They can be made intuitive by appealing to geometry, but I don't see how to do that just by staring at the algebraic picture.

Actually, proving them without picking a basis and then showing that what you get is invariant under base change is quite nontrivial and involves throwing a lot of heavy machinery around; see for example Coffman's non-coordinate proof of the statement about the trace in http://users.ipfw.edu/CoffmanA/pdf/book.pdf

For the determinant, if you want to do everything without picking a basis, you're basically proving that the exterior power operation defines a functor on the category of finite-dimensional vector spaces, which isn't that bad to do, but you somehow have to explain what all those words mean along the way.

Nice illustration of the "row picture" of matrix multiplication. The "column picture" is equally important. This intro lecture by Prof. Strang is a really good lesson on that: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...

Also, this short tutorial on linear algebra covers the row + column pictures on the first page: https://minireference.com/static/tutorials/linear_algebra_in... (disclaimer: I'm the author)

As a gamedev who uses matrices all the time, I don't find this animation useful at all for practical work. Instead I think of matrix multiplication as creating a list of ordered operations (scale x translate x rotate etc) which is just encoded efficiently in a table, to then be sent to other parts of the program for use. You can even remove items from the list by multiplying by their inverse.

But of course when you have encoded the scale, rotate and translate into the matrix, it isn't possible to decode it back out to your original values. This is obvious if you consider the fact the following produce identical matrices (just consider 2D for now):

1. Scale in X and Y of -1

2. Z rotation of 180

Yes, I know, what is your point exactly? When I say 'list of ordered operations' I mean that if I were to apply the matrix to any object, you can imagine each transform in the list being applied one after another to the object visually, resulting in the final position.

This is sounding increasingly like the Monad situation, where everybody has their own incompatible approach for explaining it, and none of them make any sense.

I feel like I understand Matrices _less_ now than when I started reading around here.

If by "Monad situation" you mean trying to learn a concept by just reading a single blog post then sure. Trying to learn something by scattered blog posts without engaging in material is a bad idea no matter what the problem is. For some reason nobody expects to read a blog post and understand the fundamental theorem of calculus without first learning intuition and doing calculations with derivatives and integrals. On the other hand, if you have been working through a concept and you read a blog post that gives a different perspective than you're used to, it might help.

If you want to actually learn something, then get something like a book or series of posts and work through examples and explore the ideas yourself. But having more than one intuition is useful and complaining about the fact that there is more than one way to understand something is lazy.

I have never understood Matrix Multiplication - which has always prevented me from doing cool 3D stuff.

Now I do! THANK YOU! :)

The power of visualization.

You totally don't need matrices to do cool 3D stuff though.

How I would explain it:

- First, show how to multiply a row vector by a column vector.

- Then, show how to "simultaneously" multiply several row vectors by a single column vector by placing the row vectors in a matrix.

- Now, similarly, do the same with the column vector. And voila, matrix by matrix multiplication.

im trying to understand this, and what you said appeals to me but i can't quite visualize what you mean. do you have an illustration?

I think this is overdone and does not help the intuition. two things: (1). the first step of transposing the second matrix 90 degree is unnecessary and confusing. A row vector and a column vector are different. This step confuses the two. (2). All the final elements are independent and should be calculated in parallel, the stepped anime is not necessary and does not reveal this basic fact.

This is very cool. For me I needed to know why it works not what it does.

I worked it out here for 2d transformations http://webglfundamentals.org/webgl/lessons/webgl-2d-matrices... and then expanded it to 3d, perspective, and cameras.

The UI confuses me to the point of frustration.

1. A "Previous" button would help since otherwise you have to go loop through the whole thing to see what actually happened (e.g. how exactly the first step lifts and rotates the matrix, what's getting added to what).

2. The final animation step is a transition from the patently false equation "[matrix1]=[result_matrix]" to the finally correct equation "[matrix1] x [matrix2] = [result_matrix]". This could be avoided if instead of transforming "[matrix1] x [matrix2]" into "[matrix1 plus overlay]=[result_matrix to be filled]" and then doing the 'waterfall', the equation "[matrix1] x [matrix2] = [result_matrix to be filled]" was maintained throughout, perhaps with the moving overlay slightly grayer after lifting [matrix2].

I have found the Stanford EE263 class to be possibly the best class on linear algebra. This might be due to the fact that it's technically a class about linear dynamic systems not linear algebra per se

http://ee263.stanford.edu/

Very nice, but the mental model I prefer is to think of a Matrix-Vector multiplication as the weighted sum of Matrix columns, and a Matrix-Matrix multiplication as just several Matrix-Vector multiplications (results being the columns of a new matrix).

Nice. I needed something like that when I was getting started with ML courses.

This would have been so awesome to have in college. I stumbled upon RegExr the other day (http://regexr.com) which I absolutely love. It kind of shares this spirit of showing people how to do things that appear complex at first glance, but have a sort of addicting nuance to them. Thanks for sharing.

I find this image a lot easier to work with. The animation above just seems a tad confusing:

https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_images/Mat...

I my head I see the matrices remaining in place, while the vectors from the left slowly float to the right, produce dot products, which in turn float into their destination cells in the new output matrix just to the right of the equals sign.

But that's just me.

See the very first video in Strang's MIT Linear Algebra course on YouTube.

There's two ways of seeing matrix multiplication (from a matrix POV, rather than as linear transformations).

Awesome. What about making it more clear that the "End" button will animate remaining steps? Before clicking it I thought it would have jumped to the last step.

Thank you! This is marvelous and really made me understand this process at a much simpler level. Please consider making more for other mathematical concepts :)

I have no idea what's going on there.

But it's a cool piece of art, I'm guessing that was the intention.

Appropriate use of the .xyz TLD :-)

I wish we had visualizations like this back in the 90's when I studied statistics.

I wonder if this visualization could be turned into a fun game.

Nice, now let's do it with the Strassen's algorithm

Cool visualization.

Now make one that can find inverses ( ͡° ͜ʖ ͡°)

Another one for matrix inversion would be nice.

Misrenders on Firefox for Android.

Looks fine on mine, what's the Android/Firefox version there?

Yeah, I'm aware. It annoys me too. https://github.com/staltz/matrixmultiplication.xyz/issues/1

this is more than just visualization. i can touch the computation. each step is tangible

Interesting, I wonder why Facebook has this website blocked, preventing me from posting it on my wall.

Good sign that it's time to leave Facebook. Really

Maybe the slightly unusual TLD?

I just had the same problem! I wanted to share it with the College group from where I graduated, but I could not do it.

Meh, the site is blocked at my workplace. Websense is really crummy.

Uggg, why is zoom disable on mobile devices?

Cool

I read the title and thought, oh God, not another one, the second was bad enough and I like to pretend that the third never happened...

Reply on news.ycombinator.com