I've always found the bipartite graph conceptualization of matrix multiplication the most intuitive, and especially so if you're familiar with neural networks: https://www.math3ma.com/blog/matrices-probability-graphs
What I might find a little more difficult to teach to people through this lens is the phenomenon of eigenvectors, but I suppose that's to be expected—nothing will work well for all purposes.
Why? Eigenvectors are simply inputs to the network where the output keeps its shape, that is, at most it gets rescaled, as if you had applied a uniform gain to the components, but otherwise it will be the same as the input.
This description changed that. Thank you!
- does a matrix have the same left-eigenvectors as its right-eigenvectors?
- what is the relationship between the left-eigenvalues and right-eigenvalues?
- is there always an eigenvector? when is there a complete set? how do you generalize your notion of eigenvectors so that matrices always have a complete set of them?
I am not sure any of these are intuitive here.
Notice that u and v cannot be equal, because they are not the same shape. However, if v is a right eigenvector with eigenvalue \lambda, then v^T is a left eigenvector with eigenvalue \lambda, as well. More or less what this means is that we tend to just ignore left eigenvalues and only use the right eigenvalues, because the math is exactly the same up to a transpose operation.
A matrix does not necessarily have nontrivial eigenvectors. Think about the 0 matrix here. But, if A is nonzero, then it must have at least one nonzero eigenvalue, hence one nontrivial eigenvector. This is because a nonzero matrix must have at least one nonzero row and column; however, if you construct the other rows (columns) to be multiples of the first column, you end up with a matrix with only one nontrivial eigenvalue. This also illustrates the conditions necessary for an nxn matrix A to have n linearly independent eigenvectors: the rows of A must be linearly independent.
All of this is covered in a decent undergrad linear algebra course. I would suggest either finding a video course, or getting a good book and working through it, if you want to understand these things better.
How about this matrix?
[[0, -1],
[[1, 0]]
A snippet of code producing a counterexample:
import numpy as np
import scipy.linalg as spla
A = np.random.randn(3, 3)
right_eigenvector = spla.eig(A)[1][:,0]
right_eigenvalue = ((A @ right_eigenvector) / right_eigenvector)[0]
potential_left_eigenvector = right_eigenvector[np.newaxis, :]
# if all components of the following are not the same, then it's not
# a left eigenvector
print((potential_left_eigenvector @ A) / potential_left_eigenvector)
# prints [[-1.1327836 -0.j -0.14850693-0.j -1.84397691+0.j]]The correct theorem is that every complex square matrix has an eigenvector. If we interpret your matrix as a complex matrix, then it does have two eigenvectors, namely [1; ±i]. Hence why I’d say it kind of “doesn’t count.”
(I also think that the counterexample exhibited "counts" in the way that it's intuitively very clear why it can't have a real eigenvector. The R^2 plane can be easily visualised, as opposed to C^2, and the matrix exhibited acts as a rotation of R^2, so of course there can't be an eigenvector. As you said, once you allow complex numbers, that's not true anymore, but at that point the visual intuition breaks down somewhat.)
Consider the matrix
M = [ 0.50 0.50 ]
[ 0.25 0.75 ]
> A matrix does not necessarily have nontrivial eigenvectors. Think about the 0 matrix here.
The zero matrix has _all_ the nontrivial vectors as eigenvectors.
> But, if A is nonzero, then it must have at least one nonzero eigenvalue, hence one nontrivial eigenvector.
Consider
N = [ 0 1 ]
[ 0 0 ]
I don't know why you consider the eigenvalue zero to somehow not count as an eigenvalue. Very suspicious.
> This also illustrates the conditions necessary for an nxn matrix A to have n linearly independent eigenvectors: the rows of A must be linearly independent.
Consider
P = [ 1 1 ]
[ 0 1 ]
> All of this is covered in a decent undergrad linear algebra course. I would suggest either finding a video course, or getting a good book and working through it, if you want to understand these things better.
sigh ... See this is why I sometimes feel bad about commenting here. Like, this comment thread was supposed to be a celebration of this different way of thinking about linear algebra, and then I have to deal with this stuff. Like I totally don’t think you meant to come across as condescending, but given that I have taught crash courses for struggling friends on linear algebra concepts they missed in their undergrad to get through our graduate work in physics, you know, it kind of does come across that way.
For the first matrix, maybe a simpler example is M = [1 1; 0 0]. Then [1; 0] is a right eigenvector with eigenvalue 1, but [1 0] M = [1 1].
Zero can of course be a proper eigenvalue. Probably they confused it with the fact that the zero vector typically doesn't count as an eigenvector.
OTOH, it is just vectors in a many-dimensional space. We're neigh-supernatural machines for describing vectors, that's how we throw things better than any other animal, I don't really know why we need analogies here. Why do we want x'x? It maps to a distance...