Math Academy, part 1: My eigenvector embarassment
frankhecker.com
frankhecker.com
A few years ago, with ample free time, I decided to refresh my (nonexistent) memory by watching online linear algebra lectures from various professors. I was surprised by their poor quality. They lacked motivation and intuition. Khan Academy offered no improvement. Then, someone recommended Linear Algebra Done Right (LADR). I read it three times, and by the third iteration, I finally began to appreciate the beauty of the theory. Linear algebra is a purely algebraic theory; visual aids are of limited help. In short, if you have the time, I recommend reading LADR. Otherwise, don't bother.
It took me time to study each page, to understand the examples, and then to attempt the exercises. It seemed very beautiful.
Then one day I came to a part I couldn't understand: I didn't see how something Axler said followed from the earlier stuff on the page actually followed. I scratched my head for a couple of hours, which is much longer than I'd spent on any previous page.
Eventually I asked a colleague for help. I showed him the page. He asked me to explain what I didn't understand. I started to explain what I knew, and how I didn't understand how this thing followed. As I was explaining it, that part suddenly clicked.
But I got stuck a few more times and didn't persevere.
I wonder whether it would have been better for me to have studied some numerical approach to linear algebra (like Strang's videos) first, rather than going straight into a book that's so abstract and proof-based.
I suppose it depends on your mathematical background.
(Your comment made me think about those folks who were once fit and muscular, then years later they are out of shape, and then they decide to get in shape say how easy it was to get back in shape. They don't realize that part of what made it easy is that they were once in shape, and they still more muscle cells or whatever.)
I don’t think any of that numerical approach helped when I read LADR. LADR isn’t about “doing the work” it’s about “doing the work to understand”. Similar to your experience I remember reading the first chapter and then among the first chapter questions I saw questions that looked like they had no basis whatsoever in what I thought I had just learned. Then, eventually, it clicked. That’s, frankly, the only way it works with Axler, so if you want it, you’ve got to do it.
My advice is to not waste time with the numerical approach and just do it.
I had a professor who used to say “being a student is suffering” but he used it to justify a bunch of bullshit. In this case, though, I’d agree with him. LADR is suffering d followed by satisfaction (and rinse and repeat).
Hilbert Space is the mathematical framework to describe QM systems.
In Europe (at least in certain countries, can't speak to all of them), maths lectures will typically be abstract and proof-based from day 1 - at least for maths majors (but frequently for CS and physics students too). Other majors, such as economics and maybe engineering, may get their own lectures that tend to be more hand-wavey because they don't necessarily need the axioms of real numbers to take a derivative here and there.
My linear algebra course was algebra and proof based to the extent that maybe a little bit more geometric intuition would have helped.
If you think linear algebra is something geometric, like "a 3x3 transform matrix is rotation and scaling; an eigenvector is something after transformation and parallel to its old self..." you will be surprised at how little LADR talks about these.
On the contrary, the most important part (imo) of 3b1b is that it helps you intuitively get these geometric interpretations.
I am not too familiar with the pedagogical history of linear algebra, but I've been reading some advanced undergraduate geometry texts from the 30s-60s and linear algebra was generally not an assumed prerequisite. There was a particular separation between the studies of "two and three dimensional vector spaces over R" (largely geometric) versus "finite dimensional vector spaces over a field" (entirely algebraic), and determinants were presented directly as volume computations. These days undergraduates mostly treat R^2 and R^3 algebraically, maybe at the expense of geometric understanding. (E.g. Euler's rotation theorem is easily proved when restated as a theorem about matrices over R^3 with determinant +1, but Euler's original statement and proof using spherical trigonometry is deeper.)
And given that most of basic QM was formalized by 1930 and relies upon eigenvectors, hard to see any physics course taught since that time not having it.
I was introduced to eigenvectors in a math course on linear algebra. They seemed esoteric but I could prove theorems and stuff… cool but kind of forgettable.
Then I took quantum mechanics. That’s where I learned eigensystems. That’s where their utility and beauty were beaten into me, problem set by problem set. In quantum mechanics, eigensystems are ubiquitous: from using ladder operators to solve the harmonic oscillator in an elegant way, to what quantum numbers actually are, to the reason behind the Heisenberg uncertainty principle, and to the so many different ways to use perturbation theory to explain atomic and molecular spectra.
You can do the basics of quantum mechanics without explicit linear algebra, and many intro physical chemistry texts aren’t able to assume the math as a pre-requisite and have to do that. But it’s tedious and awkward, like trying to learn physics without calculus.
I had never heard of them until I was _years_ into software engineering. I think this is more common than you may think. I had never dealt with linear algebra in a formal setting, despite leveraging a lot of the concepts, until then.
Jason also coined the term "Luck Surface Area" which has since been popularized by a number of others.
I haven't used Math Academy myself (although it's something I intend to try one of these days), but I can safely vouch that Math Academy isn't a fly-by-night shallow edtech grift. They've spent a small fortune and thousands of hours developing and refining content and curriculum. Math Academy is a thoughtful, intentional, well-manicured solution.
I'm using mathacademy and I will unequivocally say its better and worth the money. First it uses a assessment to test your knowledge and will send you modules to fill in the gaps. I finished foundations 1 and now I'm in the middle of foundations 2.
One thing I love about math academy is that you spend very little time reading and no time watching videos on the subject. You get a short walkthrough of how to solve a problem and then it gives you a few more problems building up the complexity. A few days later it gives you the problems again to test your knowlege. the interface is not as pretty as khan academy but you're basically learning by doing and its very effective. I wish it was around when I was in university.
Also, if they ever did a Math for Computer Graphics course I'd never cancel my subscription.
Edit: Wrt to computer graphics, going through M4ML and the ML sequence will really help you understand what's happening there. Convolutional Neural Nets, Gaussian Splatting, all rely on these same principles.
I just started going through Math for ML and can also attest, it is amazing. Pedagogically revolutionary, and helping me fix up all sorts of gaps in my math background.
It has you overfit on the style of questions they ask, and I never felt like I got a good grasp of lots of the later topics despite passing my reviews and quizzes no problem.
I’m taking their Math Foundations 2 now. So far my only two complaints are:
1. Everything feels like a random grab bag of “tricks”. There isn’t a coherent presentation of why this works or why it’s important. 2. Geometry lessons just suck. I don’t blame MathAcademy for this, since they basically follow what would be considered standard American high school geometry curriculum. But after having gone through Euclid’s Elements, everything here just feels empty by comparison.
Overall I’m still very happy with the product. They do largely deliver on their promises.
https://worrydream.com/refs/Lockhart_2002_-_A_Mathematician'...
I'm an adult and not a kid, but wrote about my experience after 100 days of using it daily here: https://gmays.com/math
The Math Academy team (including the founders) are also active on X/Twitter: https://x.com/_MathAcademy_
And there's a Math Academy community on X here in case you want opinions from other users: https://x.com/i/communities/1833198423593431339
We also tried some other programs like Art of Problem Solving (great program, but required very synchronous classes which were hard to fit in)
My suggestion would be try it for a few months,
I'm a self-motivated adult learner, so I don't know what it's like for kids. Though the program was originally designed for them, so I suspect their experience would broadly be similar to mine.
As other commenters have mentioned, you need to be okay with grinding through problem sets with no videos or UI pizzazz -- maybe this doesn't work for everybody. I'd compare it to the difference between trying to learn a language through scattered YouTube videos and Duolingo versus tandem and grinding on a good Anki set.
NB: I'm taking it for the Math for ML track and am currently most of the way through the Math Foundations III course. So I can only comment on the lower level courses.
For many, I would recommend Khan Academy. It's a great resource, especially since it's free. But if learning math is about more than just passing a class, Math Academy is worth every penny.
But as far as I remember, it hardly teaches you how to write proofs, so how much it actually teaches math is a bit questionable.
Proofs are coming, the site is a work in progress.
As someone that has the same end goal (but probably 2026 for me) - isn't it maybe wiser to do MVC before LinAlg?
Read the whole thing now, slightly disappointed OP doesn't try to tell us what an eigenvector is, based on his current progress.
In which case, I don't think it makes sense to do multivariate calculus before linear algebra.
The derivative of a multivariate function f: R^n → R^m at a point x is a linear map L: R^n → R^m so that f(x + v) = f(x) + Lv + o(|v|) for small v.
That means that multivariate calculus is about approximating nonlinear functions using linear ones in a small neighborhood, which enables you to apply tools from linear algebra to it.
You can kind of do multivariate calculus without linear algebra by essentially treating f as a collection of m × n univariate functions that you do ordinary calculus with (lots of partial derivatives) but I doubt it would be very enlightening.
https://drive.google.com/drive/folders/1JrMp7R4j86tMzHn0Sfa_...
Going back through the foundations courses on Mathacademy (I started halfway through Math Foundations II, currently nearing the end of III) has been great. It's been surprising how much I've forgotten, but also reassuring how quickly it comes back. My plan is to move on to the more advanced courses with firmer foundations.
The focus on answering questions constantly helps me focus, although the multiple choice structure is kind of limiting, if inevitable. It's frustrating to have it throw a whole load more questions at you because you missed a minus sign, where a proper teacher would have seen your working and been able to tailor their feedback.
> The most notable of these are the synthetic division method for polynomials, the various trigonometric identities, and differentiation of products and quotients of functions.
So he learned nothing you already know at 15. Or younger in Asia.
I think he forgot his goals because it doesn’t even mention eigenvectors.
I am surprised because it is not a difficult thing to understand? It is a vector that when multiplied to a matrix (which in almost all cases would change the direction of the vector), in fact only scales it - and does not change its direction.
The scale factor is its eigenvalue.
So if you hav [[2,0],[0,3]] this should when multiplied to a vector give you [2x,3y]. But if you supply the vector [1,0] or [0,1] you see that the result multiplies that vector by two. So any multiple of these eigenvectors (e.g. [10,0]) will result in a doubling of the vector.
This is not a difficult concept. By any means.
Now, do the part with the explanation of what it actually means for a (physical) system to have eigenvalues, and what it tells you about the response of such a system to external or intensive inputs, or how to change such a system to targeted a certain response.
Eigenvalues in an oscillating system describe its resonant frequencies. Its eigenvectors can describe motion at that certain resonant frequency. Imagine a bridge. If wind or traffic match a resonant frequency (eigenvalue) it would be dangerous. Engineers can redesign it to change the corresponding eigenvector and shift the eigenvalue (its resonant frequency for that mode of oscillation) to a safer range. See that bridge in London.