Matrix Multiplication

If you put matrix B over (or under) the Result matrix, the relationship between rows in A, columns in B, and elements in Result would be even more directly visualized.

You could even show the intermediate products to be summed as a diagonal between matrices A and B as you step through them. Also with direct visual alignment.

Isn’t that pretty much the same as the dot-product explanation?

I always start with Ax, which is just a linear combination of the columns, like the first figure here https://eli.thegreenplace.net/2015/visualizing-matrix-multip...

For those who want to make their own cheat sheet:

Matrix $\times$ vector is linear combination of the columns:

\begin{align}

    Ax &= \begin{pmatrix}

    a_{1,1} & a_{1,2}\\

    a_{2,1} & a_{2,2}\\

    a_{3,1} & a_{3,2}

    \end{pmatrix}

    x_1\\x_2

\end{align}

Intuitively, this example maps a point in $\mathbb{R}^2$ to $\mathbb{R}^3$

Another way to see $Ax$: Columns of $A$ form a basis, and $x$ is a coordinate vector in that basis.

Slick animations, but it needs to be slower and simpler.

In my experience, math is taught by people who are good understanding abstract concepts. It's hard for me to pick up abstract concepts, so a lot of traditional math education / exercises / tutorials work poorly for me.

Matrix Multiplication - https://news.ycombinator.com/item?id=24141472 - Aug 2020 (1 comment)

Matrix Multiplication - https://news.ycombinator.com/item?id=13036386 - Nov 2016 (131 comments)

Matrix Multiplication - https://news.ycombinator.com/item?id=13033725 - Nov 2016 (2 comments)

Posts like this seem to get upvoted on HN just because "oooh, a pretty animation" and/or they struggled with college math "See? This is how they should've taught it! Pictures are better than lectures!"

The operation you're describing is nevertheless equal to A^T B. In other words, it can be expressed using a combination of matrix multiplication and matrix transpose. I don't see what it could be used for, though.

Look: consider two functions, f(x, y) = (x + 2y, 3x + 4y), and g(x, y) = (-x + 3y, 4x - y). What's f(g(x, y))? Well, let's work it out, it's simple algebra:

f(g(x, y)) = f(-x+3y, 4x-y) = ((-x+3y)+2(4x-y), 3(-x+3y) + 4(4x-y)) = (-x + 3y + 8x - 2y, -3x + 9y + 16x - 4y) = (7x + y, 13x + 5y).

Whew, that was some hassle to keep track of everything. Now, here's what mathematicians typically do instead: they introduce matrices to make it much easier to keep track of the operations:

Let e_0 = (1, 0), and e_1 = (0, 1). Then f(e_0) = f(1, 0) = (1, 3) = e_0 + 3 e_1, and f(e_1) = f(0, 1) = (2, 4) = 2 e_0 + 4 e_1. Thus, mathematicians would write that f in basis e_0, e_1 is represented by the matrix

[1 2] [3 4]

so that when you multiply it by the (coulmn) vector [x, y], you get

      [x]
    * [y]   
 [1 2][x + 2y]
 [3 4][3x + 4y]
 

[-1 3] [ 4 -1]

Now, let's multiply matrix of f by matrix of g:

      [-1        3]
    * [ 4       -1]
 [1 2][-1*1+2*4  3*1-1*2]  = [7  1]
 [3 4][-1*3+4*4  3*3-1*4]    [13 5]

       [x]
     * [y]
 [7  1][7x + y]
 [13 5][13x + 5]

The conclusion here is that matrix multiplication is what the function composition forces it to be.

The longer answer is that matrix multiplication is essentially 2 different operations. Consider the linear funtions:

  f :: R3 -> R3
  h :: R3 -> R3

  x :: R3

The product [f][x]=[f(x)] then represents the result of a function application, while [f][h] = [f∘h] represents function composition.

For the function application portion, there is no problem with your proposal. We simply represent the point as a 1x3 vector instead of a a 3x1 vector (or similarly transpose the convention for representing a function as a matrix).

The problem comes with the function composition use-case. For your proposal to work, we would need to transpose only one of the two matrices, which means that the matrix representation of a function is determined both by the basis vectors, and a transposition parity bit. With multiplication as composition only making sense when the transposition parities don't match.

In some cases making a matrix in physics is a quite sterile representation, like you suggest, like plugging in the numbers in an otherwise generally stated question. The matrix values depend on choices of coordinate system and everything, quite inelegant in that way.

Why practically use matrixes on the computer? It's efficient to compute in blocks instead of one by one.

Suppose you have two linear functions:

  f :: R3 -> R3
  h :: R3 -> R2

  x :: R3

Similarly, a point in Rn can be represented by exactly n real numbers. Again, this representation depends on a choice of basis vectors.

For simplicity, let [?] represent the matrix representation of ?.

The second problem you want to tackle is how to compute function application y = f(x). Given the representations defined above, y is the point in R3 that is represented by the result of the matrix multiplication [f][x].

The third problem you want to solve is how to compute the function composition g = f∘h. Again, g is the function given by the matrix representation [g] = [f][h]

Unfortunately, engineers insist on designing devices which are neither perfect spheres nor perfectly flat planes (ridiculous!), and they might only have equations to describe how properties change in time or space. In this case, it can be easier to discretize the thing into a mesh, and use a matrix to describe how the physical phenomena at the points of that mesh relate to each other.

[[1,1],[1,-1]] @ [[x,y]] = [[2,3]]

@ is of course matrix multiplication but what it does does not really matter. Only that it performs some "function". But is this notation helpful? It turns out this function is really useful to answer other questions. From here, we can go on to show very interesting things like how to find the answers for x and y. It turns out that it has a recursive process similar to when solving it by hand (isolate one of the variables, plug into another equation, isolate another, ...) and part of this recursive process has a pattern we will call the determinant. Turns out that when the determinant is 0, funny things happen. The point is that it all started by trying to solve these equations which are interesting from a mathematical point of view but also show up naturally in many business applications. See Leontief's Model [1] for how this could be useful. Consumption equals production and bills being linear (#apples * $apples + #bananas * $bananas + ... = total spent) make linear systems arise naturally. All the things that come later, eigenvalues, SVD, etc. were all discovered as mathematical curiosities and later realized that they had very useful applications such as solving OLS and compression.

[1] - https://www.math.ksu.edu/~gerald/leontief.pdf

Optimizing for simplicity of procedure can make computers quite happy, and reduce errors by humans (they can still make arithmetic errors, but will make fewer errors of procedure)... and also lets you build reusable optimizations to that simple procedure (math theory about matricies) that can then be shared across multiple domains (a lot of work has been put into optimizing e.g. sparse matricies.)

Figuring out an intuitive understanding of matrix values can be done, although it depends on the problem domain you're applying them to. My problem domain of choice is computer graphics. Rotating a vertex of a triangle can be done using a 3x3 matrix:

                    [ ux_x, ux_y, ux_z ]
    [v_x, v_y, v_z] [ uy_x, uy_y, uy_z ] = [o_x, o_y, o_z]
                    [ uz_x, uz_y, uz_z ]

    v_{...} is an input vertex position
    o_{...} is an output vertex position

   ux_{...} is where a unit x vector (1,0,0) will end up at
   uy_{...} is where a unit y vector (0,1,0) will end up at
   uz_{...} is where a unit z vector (0,0,1) will end up at

   [v_x, v_y, v_z, 1.0]

    [ ux_x, ux_y, ux_z ]
    [ uy_x, uy_y, uy_z ]
    [ uz_x, uz_y, uz_z ]
    [  t_x,  t_y,  t_z ]

EDIT: Adding a 4th column becomes useful for projection matricies in a way I don't have a super great intuition for... but it also lets us multiply transformation matricies together. This is quite useful: We can simplify "transform this vertex from the original mesh, into it's position relative to this larger model, into it's position in worldspace, into it's position relative to this camera" to "multiply against this single 4x4 matrix that we constructed using matrix multiplication."

* https://en.wikipedia.org/wiki/Einstein_notation