Eigenvectors from eigenvalues
terrytao.wordpress.com
terrytao.wordpress.com
[1] https://www.reddit.com/r/math/comments/ci665j/linear_algebra...
> Apparently this relationship was discovered in 2014 by a dutch mathematician or physician. https://arxiv.org/abs/1401.4580
https://old.reddit.com/r/math/comments/ci665j/linear_algebra...
I don't know anything about the subject, I'm curious what they mean by that.
edit: The apperantly renamed it Network Architecture & Services since I finished my education.
From the article we discussed earlier, https://www.quantamagazine.org/neutrinos-lead-to-unexpected-... https://news.ycombinator.com/item?id=21528425
and gave 3 different proofs!
Suppose you have a learning algorithm that requires the eigenvectors/eigenvalues for a large matrix. Suppose new data is streamed in that increase the size of the matrix. well, now you have an algorithm for updating your existing solution instead of needing to compute the entire thing from scratch. (consider updating principal components over time for example).
Suppose you want to accelerate the process of computing eigenvectors for a large matrix. Well, you can now do parallel reduction relatively trivially given the theorem.
*edit: wording.
Eigenvalues: the Rosetta Stone for Neutrino Oscillations in Matter
https://arxiv.org/abs/1907.02534
Deep Underground Neutrino Experiment
https://en.wikipedia.org/wiki/Deep_Underground_Neutrino_Expe...
I think this result has some intriguing opportunities, and have been toying with them since yesterday’s post, but I don’t see something so straightforward.
I haven't fully worked out the math yet as well.
Now, there could be lines through the origin that map to themselves under this transformation. These lines are called eigenvectors of A, and they characterise the transformation A. (If you mirror something, a line in the mirror plane maps to itself. If you rotate something, the rotation axis maps to itself. If you stretch something, a line in the direction you stretch in might map to itself. And so on.)
Now, these lines map onto themselves, but they might be stretched or shrunk in the process - that's given by the Eigenvalue.
The famous equation is A x = lambda x: You transform eigenvector x by A, and you get back x itself, but stretched by a factor lambda (the Eigenvalue).
So, if you have some mapping of 5d space to itself, for example, it can be characterised by 5 eigenvectors (lines that map unto themselves), and 5 eigenvalues (the stretch factor for each of these lines).
Now, "traditionally", you compute the eigenvector by solving a 5d linear system for each of the eigenvalues. (5 eigenvalues, 5 slightly modified linear systems, each gives you an eigenvector of 5 coefficients, for a grand total of 25 eigenvector coefficients).
Here, it is shown that you can compute the eigenvectors of the full system without even solving that traditional linear system, but by instead taking the eigenvalues of a slightly modified smaller system. (5 eigenvalues, 5 slightly modified smaller systems M (that you get from A by striking out the i'th column and row). Each of those has 4 Eigenvalues, so you have a total of 5 + 4x5 = 25 Eigenvalues. Now, you compute some differences and products and ratios of those numbers, and hey presto, you get your 25 Eigenvector coefficients (thus the title, "Eigenvectors from Eigenvalues").
Pretty neat, though it's unclear to me how the computational complexity really compares.
EDIT: formatting, reference to title
Linear algebra for large scale "muh big data" applications has had a mini renaissance in recent years. Haven't seen any this cute and basic though.
You’ve got a matrix A, ie a 2D array of numbers.
You’ve got a vector v, that is a 1D array of numbers.
You can multiply A by v, written Av, to get a new vector.
Matrices can therefore be viewed as devices for taking a vector and giving a new one, ie as linear transformations. You can design a matrix to rotate vectors, stretch them etc.
An eigenvector of a matrix is a vector that changes only by a constant factor c, when you apply (multiply by) the matrix. So Av = kv where k is a number. For example, if k=2 then applying A just doubles every number in v. Eigenvectors are stretched or squashed by the matrix, no funny business like shearing or rotating.
An eigenvalue is the “k” mentioned above, ie there’s at least one eigenvector such that Av=kv. There could be multiple eigenvectors all with the same k, ie all changed in the same simple way by applying the matrix.
Ok? Slight complication: matrix A may contain complex numbers, not just your common garden variety real numbers. But the above all still applies.
“Symmetric matrices” are matrices that have a symmetry across the top left to bottom right diagonal. So the number at A[i][j] is the same number as that at A[j][i].
“Hermitian” matrices are the equivalent of symmetric matrices when we’re dealing in complex numbers. Rather than A[i][j] being equal to A[j][i], if one entry is a complex number then the other entry is the “complex conjugate” of the first entry.
(In case you’ve forgotten complex numbers: we extend the real numbers to including numbers of the form r + zi, where r is the “real part” (just a real number) and “zi” is a real number z multiplied by i, where i = sqrt(-1). Imaginary numbers were introduced to allow us to solve equations we couldn’t otherwise solve, but have turned out to be super useful everywhere. The “complex conjugate” of an imaginary number is obtained by flipping the sign of z, eg (2 + 3i) -> (2 - 3i).)
Anyway, so Hermitian matrices are “symmetric” in this way and turn out to be super useful in physics. They also have the property that when you multiply a Hermitian matrix H by a vector v, the possible eigenvalues are real numbers, despite the fact that H may contain complex numbers.
What the paper states is a relationship between the eigenvalues and eigenvectors of a Hermitian matrix H and the eigenvalues of a smaller “submatrix” matrix H’, where H’ is just the same as H but with a column and row removed, where the index of the row and column is the same. This is neat, because it can be used to speed up the calculation of eigenvectors (as important as say array sorting in CS) in some specific situations. Commenting further is beyond me.
Basically, this allows you to compute eigenvectors from eigenvalues of minor matrices. It is an interesting identity, but speculating on its applications is beyond my expertise.
- for the set of matrices that possess them ("transformation matrices that only perform stretch/contract"), eigenvectors (with their associated eigenvalues) play a role quite analogous to the role primes play in the integer set. They provide a unique, identifying "spectrum" for said matrix. This is made explicit by eigendecomposition (spectral decomposition).
- with extension via singular value decomposition (SVD method) to any square matrix (e.g. "transformation matrices that might also shear, rotate"), certain operations such as exponentiation of the square matrix can performed very quickly once eigenvectors/eigenvalues have been obtained via the SVD method.
Linear algebra that focuses on matrices of numbers is missing the point and a proper numerical linear algebra class will teach them better anyway
https://www.maa.org/sites/default/files/pdf/awards/Axler-For...
As a graduate student at the time, it opened my eyes about a whole bunch of theorems that I thought I already knew. And it is a lot more approachable than a textbook.
It won an award for the best piece of expository writing on mathematics in 1996. It was a deserved win. :-)
Minor nitpick: this should end with “real eigenvectors.” Rotation matrices certainly have eigenvectors, they’re just complex.
. This only applies to hermitian matrices and is therefore of limited scope.
. The identity only produces the magnitudes of the coefficients of the eigenvectors. The signs of the coefficient sare still unknown.
. Even if knowing only the magnitude of the coefficients is somehow useful, this is still very expensive to compute - just like Cramer rule which is of great theoretical interest, but has very little practical use when actually computing a matrix inverse or a determinant.
Regarding the first caveat that you bring up though, whereas the problem statement says you need a Hermitian matrix I think the results should generalize to non-hermitian matrices. In particular take a look at the third proof of the theorem at the end of the article. The only assumption required here is that the eigenvalues are simple (which does not preclude them being complex/coming in complex pairs).
Protip: I had to read the second step in the proof a few times before I could see what was going on. Explicitly what you do here is to (i) multiple from the right by the elementary unit vector e_j (ii) set up the matrix vector inverse using Cramer's rule (iii) notice that this matrix can be permuted to have a block diagonal element equal to the minor M_i with the other block diagonal entry equal to 1 and the corresponding column equal filled with zeros (iv) Write the determinant in the block form, which simplifies to the expression involving M_i by itself then finally (v) multiply by e_j from the left to get scalar equation.
That the result is a function of (n-1) minors is first-order information and would help clarify the default assumption that the result computes eigenvectors from the full Hermitian matrix's eigenvalues.
https://motls.blogspot.com/2019/11/hype-about-formula-for-ei...
https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9B... (eq. 3.6)
I haven't looked at Tao's proofs to see if this is covered, nor do I have a clear formulation of the equivalence, but it certainly is striking.
Values lambda_i(A) are introduced, but then in the formula also values lambda_i(M) are referenced for some M which is not A.
Basically, we have no idea what lambda_i(M) might mean from the statement of the theorem. Is the order lambda_1(M), lambda_2(M) ... related to the order lambda_1(A), lambda_2(A), ... How many lambda_i(M) exist? And so on.
Maybe this is all clear if you deal with minor matrices all the time, but when I see a statement like that I am not even interested in its proof anymore.
where {M_j} is the {n-1 \times n-1} Hermitian matrix formed by deleting the {j^{th}} row and column from {A}.
If I tell you f(x) = y, do you get upset that I don't define the domain, range, codomain, notion of a function, and so forth? Or if I tell you that https:// prepends a good portion of URLs on the world-wide web, would you subject it to the same scrutiny? I hope note -- to require such would be unnecessarily limiting.
My $0.02.
If you tell me that f(5) = 10, and then you tell me later on that now I can compute g(x) = f(x) - 4, for all x, from that knowledge, then I surely would wonder what you are talking about.
Now why wasn't the theorem stated syntactically correct? If we had to state it in a mechanised theorem prover like Isabelle then we would extract the definition of the eigenvalues from the statement into a separate definition, like:
For a hermitian n x n matrix H, let lambda_1(A), ..., lambda_n(A) be the eigenvalues of H.
But of course, here we assume an order of the eigenvalues, and we don't really want that, because it is immaterial to what follows. So we rather have to write it like that:
Definition: For a matrix H, let L(H) be the set of its eigenvalues.
Theorem: If H is a n x n Hermitian matrix, then |L(H)| = n.
But, of course, this is not true, as |L(H)| < n is a possibility. So you would have to account for that somehow as well in your definition / notation.
So it is probably better to go back instead again, and explain the notation as follows:
Notation: For a Hermitian n x n matrix H, let lambda_1(H), ..., lambda_n(H) be its eigenvalues (in an arbitrary order).
Theorem: What Tao said.
The notation could still not be written in Isabelle like that, but at least it has proper scoping now.
Cheers.
What might not be obvious is the ordering of the eigenvalues, which must be the same for the formula to make sense.
Part of the art of mathematical exposition is to walk the line between under and over explaining your process based on your audience.
If the intended audience is academic mathematicians or even just people with a math degree, then the notation is probably familiar. I imagine that's the target audience here.
Still, how many lambda_i(M_j) are there? From the formula I am led to believe there are n-1 such eigenvalues of M_j.