Introduction to Applied Linear Algebra Vectors, Matrices, and Least Squares [pdf] | Hacker News Reader

Introduction to Applied Linear Algebra Vectors, Matrices, and Least Squares [pdf]

web.stanford.edu

71 pointsAnon8415 comments

Please don't deep link to PDFs when there is a sensible contents page that has more information and other links.

https://web.stanford.edu/~boyd/vmls/

As a rule-of-thumb, I would stay away from any "linear algebra" textbook that does not even have definition of an abstract vector space in the first few chapters. In a sense, it's like programming without data types.

When I was in high school, I learned linear algebra from similar textbook, but it wasn't until college (and that was an engineering linear algebra course!) that I got any sort of understanding of what it's really about.

Interesting how k-means clustering is considered appropriate for chapter 4 of this 19 chapter book. I don't know anyone who even considers it related to the material in linear algebra (which is more about vectors, matrices, spectral values, and applications, whereas k-means is more an application of knowledge about algorithms and data structures). But I assume the pressure to sneak ML topics into dry fundamentals courses is inevitable.

This is a course on APPLIED linear algebra.

But k means isn't linear algebra. Might as well put k means into a book on APPLIED car repair.

I like the idea of k means in a APPLIED car repair book :-)

Chapters 18 and 19 are about nonlinear least squares. I think that ideas outside the nominal linear algebra domain (like k-means, or nonlinear LS) can be helpful in a way like sex between people with different genomes; it's useful for exchanging ideas.

Local nonlinear optimizations can be found locally by constructing linear approximations of the problem statement near the initial conditions, so that is actually connected with Linear Algebra. The issue here is that most students spend only a short time of their life getting exposed to fundamental topics like linear algebra, that's not the time to waste their attention on hype of the month

But most of these students (particularly ones taking the lin alg course that apparently has APPLIED in all caps in the title) are going to be engineers or similar who will never encounter the fundamentals outside of their application in tools like k means. Teaching them an abstract field without connections to what motivates it (intellectually or practically) is pedagogically not ideal.

>Teaching them an abstract field without connections to what motivates it (intellectually or practically) is pedagogically not ideal.

While fair in general, the point is k means has nothing to do with linear algebra besides they both use numbers sometimes. It would be like sandwiching in a chapter on RSA crypto because it's interesting and number-y but still has no connection to linear algebra.

K-means was in optimization in operations research decades before anyone heard of machine learning in computer science.

Computer science was not a prerequisite to that work in optimization.

Optimization in operations research and much of applied math more generally are awash in important algorithms without reference to computer science and, really, e.g., the Dantzig simplex algorithm of about 1949, before computer science was a recognized academic field.

Such applied math includes A. Wald's work on sequential testing in statistics, the fast Fourier transform, Gram-Schmidt orthogonalization, Gauss elimination, Gauss-Siedel iteration, the Hungarian method for maximum matching, Runge-Kutta in ordinary differential equations, and much more.

OK, but why is it in a book on Linear Algebra today? Look at any introductory Linear Algebra book more than 3 years old and you won't find it in the curriculum

Do you have a personal vendetta against clustering or something?

I don't know too much about linear algebra but I know I had to learn some when my boss wanted me to do unsupervised classification. Maybe the book just considers it a good application of the material.

What do old books have to do with what should be included in new books?

ML is hyped enough already and kids already don't know enough fundamentals. They're never going to sit in front of a Linear Algebra class again so the appropriate use of their time is to teach them as much linear algebra as possible and not to turn it into a class on today's hype train

Math textbooks include topics in physics and theoretical computer science all the time. How is ML any worse?

Besides, a lot of introductory math textbooks are thinly veiled introductions to subjects the authors think are important.

The difference is k means has literally nothing to do with linear algebra. Might as well sneak in a chapter on finite state machines, or zebras, either would be just as relevant to linear algebra, which is to say that at a certain point the topics in a linear algebra course should be somehow related to linear algebra.

Reply on news.ycombinator.com