Fundamentals of Linear Algebra and Optimization [pdf]
seas.upenn.edu
seas.upenn.edu
If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multiplication of their matrix forms, I throw the book away in disgust. I throw most books away in disgust.
This is the second book I've seen that does it right, but unlike the other one [1] this one wraps the very correct presentation of matrices in so much technical language and such a boring cover story that I almost threw it away in disgust anyway.
My greatest wish in STEM education is that we teach linear algebra better. It's such low hanging fruit and could change so much!
[1] which I saw linked in HN a year ago and don't remember the name of, but it was something like "linear algebra taught the correct way" and was apparently well known in the States so ask your friends
[1]: https://www.amazon.com/Linear-Algebra-4th-Stephen-Friedberg/...
After doing that book I wanted to see what other books were out there, which ones were the good ones, and sort of classify them. I came up with this list: https://begriffs.com/posts/2016-07-24-best-linear-algebra-bo...
I classify the books as Generalist (like Axler's), Theoretical (starting from e.g. modules rather than vector spaces), Numerical (matrix normal forms and algorithms), Practice (books full of problems), and a few other categories.
Ironically you put "Jänich, K. (1994). Linear algebra. New York: Springer-Verlag." into the "Theoretical" section: In Germany this book (its German original) is not that well-regarded, since it is considered as far to shallow. Much better (and more hard to read) German textbooks are
Gerd Fischer - Lineare Algebra: Eine Einführung für Studienanfänger (Linear Algebra: An Introduction for freshmen)
Siegfried Bosch - Lineare Algebra (and its companion book: Siegfried Bosch - Algebra)
Axler was way too abstract for me as a first linear algebra book. I look forward to returning to it once I finish Strang.
[1] https://www.amazon.com/No-bullshit-guide-linear-algebra/dp/0...
I can't quite tell if the book passes the test, since the entry-point is the definition of matrix-vector product which is very close to a "complicated formula with a couple of nested sigmas" but the book also mentions the notion of matrix representations, so hard to tell overall.
I mean, it defines matrix-vector product in a nice but abstract way, and then in the next paragraph explains why we chose that definition. And it does say "this, this here is the one important idea in this book", which it gets many points for.
I really would prefer a textbook to start with "ok, here's something we want to do. Lets figure out a formula for it. Now lets give it the name matrix-vector product".
Same for matrix-matrix products.
I don't know any really good linear algebra books, in my school it was taught the worst possible way (define a field, rote-learn the mechanical operation of matrix multiplication, talk about vector spaces for a while, talk about matrices, first mention linear transformations). Some people on this thread gave other suggestions, I'd start reading a few and choose the one I connect with the best.
Anyway, I really learned linear algebra from using it.
The two places to start historically (and easily accessible to high school students) are:
(1) understanding and working with displacement vectors in 3-dimensional Euclidean affine space and in general thinking about transformation geometry (sometime later this can be extended to other kinds of non-Euclidean or non-metrical geometry), especially with reference to problems in Newtonian mechanics.
(2) systems of linear equations: this one is inherently coordinate-heavy and matrix based, at least to start out, and explains our conventions for how matrices are written, index order, multiplication of matrices by "column" vectors on the right, the use of matrix equations to fold several equals signs into one, etc.
After that I'd call out the manipulation of vectors of polynomial coefficients as an accessible additional concrete example of a linear space.
Discussion of other linear spaces or more purely abstract treatments proving properties from axioms can come sometime later, after students have familiarity with some of those tools, and after they have applied them to some problems in statistics, multivariable calculus, computer graphics, ODEs, optimization, etc.
So basically the chain rule states that if you have two functions F, G composed together and you want to find the derivative (the linearized approximation), you simply compute the linearized function for both F and G, and compose the linearized version afterwards.
You can read the Wikipedia article on the chain rule and it eventually gives this simple and elegant formula for the chain rule: https://wikimedia.org/api/rest_v1/media/math/render/svg/8b0f...
But the unfortunate reality is that too many textbooks formulate the chain rule in such a complicated manner that it obscures the simplicity and elegance of the chain rule.
That's easy. Vector spaces and linear transformations are best understood abstractly. Matrices are best understood as visually and computationally convenient representations of tensor products, so don't mention matrices at all until well after you have established the basic properties of tensor products, in particular, the natural isomorphism between `L(V,W)` and `Dual(V) (x) W`.
I’ve seen worse though: those that attempt to shoehorn abstract algebra in the process by first rigorously defining a field.
And no, teaching multiplication doesn’t involve memorizing a formula; it’s a simple mechanical process of arranging one matrix on the left the other at the top and multiplying/adding their corresponding rows and columns. Once this process is familiar to a student, they will have no trouble writing out the formula with nested sigmas.
In finite dimensions, linear transformations and matrices are exactly the same object mathematical objects, with very different notations (matrix notation (boxes with numbers inside) vs the linear space/linear transformation notation). I would rather the students to learn deeper mathematics only in matrix notation, rather than to master less substantial mathematics with both notations.
Teaching both notations may reinforce the idea that matrices and linear transformations are different mathematical objects in finite dimension. Teaching the more abstract notation is mainly useful in infinite dimension (Hilbert spaces).
Whether you're discussing the Jacobian of a function, or change of basis matrices, learning the matrix formula is a lot less useful than seeing how it falls out of the linear function definition.
The formula is hard to memorize and gives no intuition for why anything is true. But from the linear function definition it is easy to reconstruct the formula.
In fact this is so true that I would say that anyone who only knows the matrix definition does not actually understand linear algebra.
They are not: Matrices represent linear transformations with respect to a given basis. Linear transformations are completely independent from any chosen basis.
I've had a quite accomplished mathematician for a professor who was very adamant that students should learn about basic matrix operations before the abstract theory. According to him, his undergraduate education at Princeton did things the "right" way, but left him quite confused about the main ideas and motivations. He ended up making some quite important contributions to the theory of representations of some certain algebraic structures if I recall correctly.
Seconding your wish that curricula would lead with motivations and then drill mechanics rather than drilling mechanics for 11 years and finally giving you the motivation in year 12.
I found it easier to learn things like stats, linear algebra, and calculus once I had a personal application to it. Even something that I love in Reinforcement Learning, I would sometimes get sleepy reading pages and pages from the Sutton and Barto book, but as soon as I worked on a coding exercise, I could spend hours actively participate with trying to solve the problem.
As a mathematician I say:
What is considered as interesting is different for each person. I, personally, for example deeply love this really abstract stuff (but of course I am aware that other people have different preferences). So I would say even finding "interesting stuff" that many students in the lecture hall might be interested in is really, really hard.
Another important argument against your idea is: To be even able to formulate the ideas from AI, one first have to learn and understand the words of the language in which one will formulate this. And these words are like "vector space", "linear map", "tensor product" etc. and understanding their meaning means knowing theorems about them. Starting with advanced topics, such as your AI example, is like giving beginner language learners a really advanced text in the foreign language that they just begin to learn. In other words: A reallz dubious idea.
Considerung your post I can only ask you why you did not have a talk with the course advisor of your faculty. He would immediately have told you why these lectures are important for the things that you are actually interested in.
As a follow-up I would recommend Trefethen & Bau, Numerical Linear Algebra.
There's so much hype about machine learning but so few people seem to appreciate the fundamental importance of linear algebra (especially in ML).
This fast.ai course is also great: http://www.fast.ai/2017/07/17/num-lin-alg/
Of course it won't give you the math background, but you can abd should pick it up after.
It is one of the class I enjoyed the most when I was there. Some of his other books are quite enjoyable as well.
Dr. Gallier's PhD work was focused on logic, but his current research has a lot to do with computer graphics and computational geometry. He spent a long time study graduate-level algebraic geometry and algebraic topology while having the duty to be a professor at computer science department. His stories is always an inspiration for me to learn more.
The list of guides is here: http://neanderthal.uncomplicate.org/articles/guides.html
And remember, linear algebra has broader application than just geometry!
Looking at page 25 of TFA:
> In Definition 1.2, the field R may be replaced by the field of complex numbers C, in which case we have a complex vector space. It is even possible to replace R by the field of rational numbers Q or by any other field K (for example Z/pZ, where p is a prime number), in which case we have a K-vector space (in (V3), ∗ denotes multiplication in the field K). In most cases, the field K will be the field R of reals.
(0) There is way more to linear algebra than linear programming.
(1) A linear algebra course is not the right place for an extensive study of how to solve optimization problems anyway. That belongs in a real analysis course.
> Prerequisite(s): Undergraduate course in linear algebra, calculus
> The goal of this course is to provide firm foundations in linear algebra and optimization techniques that will enable students to analyze and solve problems arising in various areas of computer science, especially computer vision, robotics, machine learning, computer graphics, embedded systems, and market engineering and systems. The students will acquire a firm theoretical knowledge of these concepts and tools. They will also learn how to use these tools in practice by tackling various judiciously chosen projects (from computer vision, etc.). This course will serve as a basis to more advanced courses in computer vision, convex optimization, machine learning, robotics, computer graphics, embedded systems, and market engineering and systems.