Hacker News Reader

Comment by mikhailfranco | Hacker News Reader

The title words remind of an unrelated fact, Gershgorin Disks:

  The eigenvalues of any N x N matrix, A, 
  are contained in the union of N discs in the complex plane. 
  The center of the i_th disc is the i_th diagonal element of A. 
  The radius of the i_th disc is the absolute values 
  of the off-diagonal elements in the i_th row.

https://blogs.sas.com/content/iml/2019/05/22/gershgorin-disc...

It's rather remarkable, unexpected, and perhaps shocking, when you first hear it. There does not seem to be enough information to make it true. But it is.

I used it in a real project to cancel noise.

View on HN

Matrix theory by Franklin is a great, affordable book containing many interesting results such as this — can highly recommend for those interested in linear algebra.

https://www.amazon.com/Matrix-Theory-Dover-Books-Mathematics...

If I have an atrophied high school level understanding of linear algebra, will I get anything out of that book?

Certainly a useful reference, if you might deal with the topic for some project … but not the best refresher out there for the basics.

For the basics, start here:

https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010...

https://store.doverpublications.com/products/9780486411798

You may enjoy this:

https://bwlewis.github.io/cassini/

Any more details on this one? What might one use-case be for this application?

Matrix Analysis by Horn and Johnson is another great book, though it is a bit pricier than a Dover book.

It's not that remarkable at all? The proof requires the definition and triangle inequality, that's all?

Given Ax=λx, take i for which |xᵢ| is largest. Look at the i'th equation: sum aᵢⱼxⱼ = λxᵢ, move the aᵢᵢxᵢ term to the rhs, take absolute values, divide by |xᵢ|, apply triangle inequality, and you have |aᵢᵢ - λ| ≤ sum |aᵢⱼ| over j≠i. So for every eigenvalue you can find such a disc.

That's by column, for row use Aᵀ.

Being easy to prove doesn't make it unremarkable. Lots of theorems, including this one, have straightforward proofs once you are given the exact formulation. The tricky part is coming up with the idea for the theorem itself. I remember being (mildly) shocked when I was taught this in undergrad, it just seemed too good to be true.

I think that's just how textbooks present it. A fact is stated but you're lacking intuition.

If you had played around a bit with Laplacian matrices, like tri-diagonal matrices with stencil [-1, 2, -1], and found that its eigenvalues are within 2 ± 2, and if you also realized that A + τI has the same eigenvalues shifted by τ, then it's a small step to consider that the magnitude of the off-diagonal may have something to do with the spread of eigenvalues.

It's likely that Gerschgorin stumbled upon it like this.

> tri-diagonal matrices with stencil [-1, 2, -1],

Just trying to understand these terms. So is a 5x5 tridiagonal matrix with your stencil look like this?

     2 -1  0  0  0
    -1  2 -1  0  0
     0 -1  2 -1  0
     0  0 -1  2 -1
     0  0  0 -1  2

Yeah, exactly. Your matrix shows up when you do a 3-point discretization of -u''(x) = λu(x) with u(0) = u(1) = 0, at 5 equidistant (interior) grid points. That's the discrete, 1D version of the problem they're looking at in the paper.

See https://en.wikipedia.org/wiki/Compact_stencil#Three_Point_St...

Just thinking about some intuition for the result of the theorem: If the off diagonal elements are zero then the diagonal element is an eigenvalue, by continuity of the determinant, if the off diagonal element are small then $det(A-a_{ii}\lambda)$ is almost zero, that is the new eigenvalue is near aii. So it suggests that the off diagonal elements measure how far is aii from being an eigenvalue.

The Euler's identity is also trivial to deduce, but this doesn't diminish its beauty.

The Yoneda lemma is another great example of (once you've got the right setup) trivial but beautiful

Huh?

Math 101, simple is better.

removed

They are saying that the Greshgorin theorem that the OP is talking about is simple to prove, not the Polya conjecture that took 70 years from the article.

What was the noise cancelling project? How did you use this fact to cancel noise?

Just guessing, but....

A common noise cancellation technique is to throw away small eigenvalues, as in PCA. This result relates eigenvalues to the structure of the matrix, so might be helpful for reducing ev's without bothering with diagonalization?

[Edit] This would presumably involve just zeroing out the rows with small diagonal elements and small-ish off-diagonal norm... Center the eigenvalue estimate disk at zero, and then zero out the rest of the row to make the estimate exact.

(and after one more thought about it, one would zero out the row /and column/ to preserve the symmetry of the matrix, if applicable, and thus keep the eigenvalues real. and one would probably want to think a bit about whether this kind of deletion actually makes sense for the problem... doing real PCA isn't /that/ hard in most cases - I think I would do something this janky only for extremely large matrices or for realtime operation on a microcontroller or something, and then only after thinking hard about it.)

Sounds very cool. One thing I didn’t understand though:

> The radius of the i_th disc is the absolute values

How can a radius of a single disc (i.e. a single value) correspond to multiple values?

I’m assuming, but could be wrong as my matrix math ended with GameDev, that the radius of the disk = the absolute values of the other things. +\- depending on which side of the disk they fall?

That line got me as well.

It's missing the word "sum". Which apparently my brain auto-deduced for me because I didn't notice anything off in the sentence on the first reading.

My brain also automatically added sum and also haven't noticed anything, but I have a math degree, maybe it's just assuming things :-)

I have a physics degree and did not assume it, that might explain the difference :)

If you look at the link there’s a formula showing it’s the sum.

Specifically:

    r_i = \sum_{i \ne j} |A_{i j}|

Sum the absolute values of a row for all the off diagonal elements

An intuitive explaination is imagine a nearly diagonal matrix where the values along the diagonal are much larger than values on the off diagonal. We know the eigenvalues of a diagonal matrix is simply the values on the diagonal, so for nearly diagonal matrices you can be pretty sure that the true eigenvalues are going to be pretty close to those diagonal entries, but a natural question to ask is how far we'd deviate from those diagonal entries.

The answer to the above question is Gerschgorin disks and it's closely related cousin Brauer's Oval of Cassini.

For matrices with real eigenvalues it's moreso along the real number line, only for cases where the eigenvalues are imaginary do we imagine disks.

It has an intuitive element to it; we’re looking for the eigenvalue, the vector/scalar pair where the scalar has the same effect on the vector as multiplying by the matrix. And we’re comparing against something that looks vaguely like the magnitude of the matrix (shifted by the diagonal, and of course if you had a diagonal matrix, the eigenvalues would just be the diagonal).

Not a proof or anything, of course the proof is on Wikipedia and nice and elegant. Just a thought on the gut feeling.

I agree that it is a very nice result.

bee_rider already touched on this in another comment, but the theorem makes sense if you consider a matrix with large diagonal values and small off-diagonal values (in magnitude). If I have a matrix with 1,000,000 on the diagonal and 1 everywhere else, I'd expect the eigenvalues to be 1,000,000 plus or minus some small error. The Gershgorin disk theorem proves this and puts an upper bound on the error.

The diagonal elements of matrices have a lot of rather "magical" properties if you think about it. Their sum is also the sum of the eigenvalues of the matrix. And if you have a matrix A that is singular, you can choose any value x that is not an eigenvalue, and then A - xI is invertible but still mostly behaves like A.

See also Brauer's oval of Cassini which give an equivalent or even better approximation of the eigenvalues of a matrix.

Reply on news.ycombinator.com