An Intuitive Derivation of Eigenvectors | Hacker News Reader

An Intuitive Derivation of Eigenvectors

dhruvp.netlify.com

209 pointsdhruvp26 comments

If you want to get an intuitive, visual understanding of linear algebra -- including eigenvectors/eigenvalues -- 3blue1brown's playlist on the subject is just ... perfect. https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...

Problem with visual understanding of linear algebra is that it is no understanding at all when it comes to linear spaces over the complex numbers (which are an important tool in quantum mechanics, for example) or infinite-dimensional spaces. Attempts (or the habit) to use the intuition gained when being exposed to an elementary examples often lead to gross misunderstanding and logical errors.

It seems like you're suggesting the added complexity of complex numbers makes visual understanding unhelpful; do you care to explain why visualizing complex numbers or multidimensional spaces is so impossible?

really good explanation. I like it even better than 3blue1brown or visualisations that I had seen.

It is better because it really covers every step of the construction process.

And offers explanation of why certain thing are not the right construction blocks. The author gives a visual example, for example, of why basic vectors 1,0 -1,0 are bad. The article shows they cannot span the whole space.

Those kinds of explanations of 'bad constructions' are difficult to show in visualizations, that show 'good' constructions only.

But, yet, in my view, these negative examples, are really helpful to explain the material that otherwise, requires 'intuition'.

Not everybody has same intuition, so showing negative examples/impossible constructions, and why those do not work -- is a good way tuning one's intuition.

---

On a separate note, I am wondering if such good step by step + counter examples, knowledge presentation -- is a result of author studying at MIT, or a natural trait (or both) ?

I think what happens is when you spend enough time thinking about explaining it to others or yourself (for whatever reason), you tend to be able to simplify explanations and find creative ways to explain it. Obviously, it is a mixture of raw talent and something else but there is enough raw talent in this world and not enough something else.

You can absolutely visualize why [1, 0] and [-1, 0] is not a basis for R^2! The problem is if you plot them in R^2, then they and the origin all lie on a single line. No matter how you sum copies of these vectors, you can never "escape" this line. Thus, you cannot construct vectors in R^2 that do not lie on this line.

Yes, that's exactly how the author of the article is explaining it (including the picture).

I was just commenting that it was good that, the author took time to explain this.

And those kinds of nuances/counter examples are not always available in explanatory dynamic visualization tools (such as the ones referenced in the comments here [1] ).

To be clear, I am not arguing against dynamic visualizations.

These are very nice, very very helpful, and very time consuming to build and present on a website.

I am just suggesting that showing negative examples is a complimentary, and powerful explanation technique.

[1] http://setosa.io/ev/eigenvectors-and-eigenvalues/

> The author gives a visual example, for example, of why basic vectors 1,0 -1,0 are bad. The article shows they cannot span the whole space.

I'm assuming you didn't mean the word "basic" this way, but on the off chance you did: {(1,0), (-1,0)} doesn't just not span R^2; it's not a basis of R^2. Partly because it doesn't span R^2 like you've said. But it's also linearly dependent, which is really the more important thing (since every linearly independent subset of a vector space whose cardinality matches the dimension of the space is a basis).

Just wanted to make the terminology of "basis" clear, since you used the term "basic" which could somewhat collide with it.

> On a separate note, I am wondering if such good step by step + counter examples, knowledge presentation -- is a result of author studying at MIT, or a natural trait (or both)?

MIT has very high quality instruction, and the author could be naturally talented at exposition. But I doubt it's because of either of these things. Realistically you could attend any math department in the top 100 and be equally capable of succinctly writing about this topic at the same detail for a general audience the way the author has. Schools like MIT shine on the upper level material and research capital.

Likewise it's probably less about natural talent and more about practice and good editing. On the other hand, what would be very helpful is getting exposure to many textbooks with different pedagogical styles. Someone who learns linear algebra in the hardcore abstract style might have difficulty presenting the material in an accessible, geometric way even if they fully understand it. Similarly if you've only learned about vectors in the Euclidean sense of direction and magnitude, it can be difficult to teach in the more abstract, non-geometric style.

Hey!

Author here. Really appreciate the kind words. If you have any feedback on what I can do to improve the explanation further, I’d love to hear it.

Also, if you’re interested, I’ve written some other posts on explaining concepts in Math and ML following a similar approach:

1. Brief History of CNN based Image Segmentation: https://blog.athelas.com/a-brief-history-of-cnns-in-image-se...

2. Understanding Baidu’s Deep Voice for voice synthesis: https://blog.athelas.com/paper-1-baidus-deep-voice-675a32370...

3. An intuitive explanation of matrices as linear maps: https://dhruvp.netlify.com/2018/12/31/matrices/

Your exposition is very good. If I could make one pedagogical recommendation, it would be that you give a gentle introduction to linear dependence and independence. You don't need this to treat the subject in the geometric fashion you're using (it's implicit in talking about calculating lines on the Euclidean plane). But if you explain it well it's a very powerful didactic mechanic for your audience, because then they can generalize the reason why {(1, 0), (-1, 0)} is not a basis of R^2 to arbitrary dimensions n. It also leads you into the neat result that any linearly independent subset S of a vector space V whose cardinality matches the dimension of V is a basis.

That being said, I know all too well what happens to good exposition when you try to shove too much into it. So feel free to ignore this; you write well and I think you accomplish the goal of presenting the material in an intuitive way.

One way to think of eigenvectors:

Your linear map A moves things around, and you aim to characterise the linear map.

So, look for lines (through the origin) that are not moved. Those are given by eigenvectors. A point on that line might be moved closer to or further away from the origin (depends on eigenvalue < or > 1), or even flipped to the other side (if eigenvalue < 0), but the line as a whole is mapped to itself.

Seriously consider buying a domain name for $8 or whatever. You can use it for free with netlify hosting anyway, and then you are in control.

I know! I’ve just been a little lazy. I’ll get a domain later today.

I was confused why/impressed at netlify was writing about eigenvectors in the first place. HN in particular only shows the top-level domain so I couldn't tell it wasn't netlify themselves until clicking around your site. :)

I also really liked this explanation: http://setosa.io/ev/eigenvectors-and-eigenvalues/

You can drag things around and change values -- if you're a visual learner, it really helps grasp things like this.

IMO eigenvectors are easiest to understand in connection with differential equations, and that's also one of their most important applications. If you plot the flow of the equation x' = Ax then the eigenvectors are visually apparent.

https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B...

The eigendirections are the directions where the solution moves in a straight line.

Not all matrices have (real valued) eigenvectors:

https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5...

The difference between an exposition in text(like this one) and videos(3blue1brown) is how people prefer to build knowledge and more deeply with how one learns. The quality of both of these is excellent. And one can test what works best like explaining to oneself(or a rubber duck) after reading/viewing the material and how one can recall things.

There is no shortage of intuitive explanations of various elementary concepts in mathematics (and physics); my personal favorites are by E. Khutoyansky [1]. (Surely enough, there is a video on eigenvectors!)

[1] https://m.youtube.com/user/EugeneKhutoryansky

I liked the MIT lecture on eigenvectors: https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algeb...

The thing I use to visualize an eigenvector is exactly that. A rotating planet. The eigenvector of the rotation matrix being the axis of rotation.

It gets weird when thinking of 2D rotations though... Too complex for me!

The eigenvalues of a 2d rotation matrix are complex numbers. There's no point in trying to visualise the corresponding eigenvectors. You'll need 4 dimensions for that. (The 4 dimensions come from the fact that the eigenvectors of a 2d rotation matrix are elements of ℂ^2, which is topologically equivalent to (ℝ^2)^2, which is equiv to ℝ^4).

Personally, my "intuition" is based on analogies with non-complex eigenvectors, and experience solving eigenvector problems algebraically without using pictures.

Also, your 3d rotation example actually has three eigenvectors, two of which are complex. You've only found the one that's real.

OP here: Your approach is pretty on point because Planetary Motion was actually the first recorded motivation for eigenvectors. Cauchy published a paper on planetary motion in 1829 which made use of what we now call Eigenvectors. See here for more info: https://www.maa.org/press/periodicals/convergence/math-origi...

Good writeup! A tad bit disappointed as I was expecting some new insights, but this is more or less standard material about eigenvectors you would get in a typical university course.

I really think some better name than eigenvectors (and eigenvalues, etc..) should be popularized. I find them to be very obtuse and opaque terms.

eigen in German means owned/proprietary. this has always been a useful mnemonic for me.

this is awesome tyvm!

Reply on news.ycombinator.com