Clojure Linear Algebra Refresher, Part 4: Linear Transformations | Hacker News Reader

Clojure Linear Algebra Refresher, Part 4: Linear Transformations

dragan.rocks

57 pointsmetalock11 comments

One meta comment.

You've taken the usual approach of explaining matrix operations as an unmotivated "this is what we do" and then tried to layer linear transformations as a weird abstract thing which somehow relates. Most people do this, and it makes sense for people who don't want to actually understand.

If you want people to actually understand, the relationship goes the other way. Linear transformations are the thing we really care about, and matrices happen to be the most convenient way to write them.

More specifically if T is a linear transformation from V to W, with bases (v_1, ..., v_n) and (w_1, ..., w_n), then then we can write down M = (T(v_1) T(v_2) ... T(v_n)). So far that's just a rectangular array of numbers, uniquely defined for the linear transformation. Now if we have a vector v = a_1 v_1 + a_2 v_2 + ... + a_n v_n then just tracing through linearity, we get that T(v) is Mv with the regular matrix multiplication. Furthermore we can now prove that any rectangular matrix defines a unique linear function with that rule.

The result? When you have 2 vector spaces with a basis for each, each linear function gives a unique matrix, and each matrix gives a unique linear function. Matrices are just convenient ways of writing down linear transformations.

OK, now if S is a linear transformation from W to U, and we have a basis for U, then what about the function S o T defined by S o T(v) = S(T(v))? If you work it out, it is a linear function, and its matrix is the usual matrix multiplication. So matrix multiplication is the same as function composition for linear functions.

Here is something fun. You know the associative law? That magically just works after a ton of calculations for matrices? Well, here is why it works. For any three linear transformations R, S, and T which can be composed, and any vector v, by definition:

((R o S) o T)(v) = (R o S)(T(v)) = R(S(T(v))) (R o (S o T))(v) = R((S o T)(v)) = R(S(T(v)))

Which means that functions under composition are associative. Since matrix multiplication is just a way of writing function composition, it must also be associative. QED!

As you move through things like change of basis matrices, it is all a lot easier to understand starting from the point of view of linear functions as being fundamental, and matrices just being a way to represent it with a given pair of bases.

I'm not so sure about that. I intentionally direct the reader to the textbook for real understanding. The aim of the article is to connect the dots from that textbook to clojure code, as is stated at the beginning. I am not a mathematician, nor I think a blog post is enough on its own to teach someone linear algebra.

It depends on your goal.

If your goal is real understanding, the approach that I stated is the right one. If you've learned it the other way and wish to actually understand it, you'll still have to learn the approach that I stated.

Your goal is explicitly not real understanding. And most textbooks do it in the order that you did. (Partly because professors think that most students aren't going to try to understand, so there is no point in giving explanation.) So you probably made the right choice.

But I would still like to see even a passing mention that matrices of just a representation of a linear function given two bases, and matrix multiplication is function composition. That could set a lightbulb on for someone struggling through a problem.

I do not know whether you've read part 3 of this series. It explicitly treats matrix multiplication as function composition, and goes with this at length.

As for the rest, you might be right or not; but I am not competent to reform world's math education. If you can write better linear algebra guides than the standard textbooks, I'm all for it. Please inform me if you do that, I'd love to read that. (BTW It's not sarcasm - I'd really love to see linear algebra explained better than in standard textbooks. So far all fancy tutorials turned to be good only as entertainment).

I had not yet read it. Having just fixed it, it is missing the role of a basis. And without it, you'll have trouble figuring out coordinate systems, changing your basis, and so on. Which gives you no way to keep straight such confusing things as the fact that if your coordinate system rotates one way, your representation of things rotates the other. (Try it! Stand up, turn clockwise and see the world spin counter-clockwise!)

Let me offer an abstract example. Consider the polynomials of degree at most 2. There is an obvious basis, namely 1, x, and x^2. A polynomial like x^5 - 3x + 2 can now easily be written in coordinates as (2, -3, 1).

However we have many other coordinate systems that might be convenient. For example suppose that we're sampling data, and can measure p(0), p(1) and p(2). How do we find what polynomial that is? Here is an easy way. We can easily find the new coordinates for our basis vectors: 1 -> (1, 1, 1), x -> (0, 1, 2), x^2 -> (0, 1, 4). That means that we can write down the matrix representing the identity transform (nothing happened), going from the basis we have, to the new coordinate system:

    ( 1  1  1)
    ( 1  2  3)
    ( 1  2  4)

That's the change of basis matrix one way. Invert it.

    ( 2 -1  0)
    (-3  2 -1)
    ( 1 -2  1)

And now we can go the other way. The polynomial that we want from our sample data will be (2-3x+x^2)p(0) + (-1+2x-2x^2)p(1) + (-x+x^2)p(2).

There are a lot of problems where linear algebra comes up that you can think through more clearly if you think about things this way (complete with the role of the basis!) than if it isn't fully digested.

As for a better book, well, I already recommended Down With Determinants! :-)

I'm not sure if you'll see this 2 days later, but I've been trying to make heads or tails of your example and I think there’s a mistake in there somewhere.

Maybe I'm hopelessly lost, but, for one, the inverse of

    ( 1  1  1)
    ( 1  2  3)
    ( 1  2  4)

is not:

    ( 2 -1  0)
    (-3  2 -1)
    ( 1 -2  1)

On the off chance you see this, any pointers?

I did not see it.

As I commented in email, I did it by hand while very tired and made multiple mistakes.

Down with Determinants is a paper, and I guess it is super-clear once you know the stuff already. For non mathematicians, and people who'd like to learn linear algebra? Right...

Anyway, who is missing the role of the basis? It's in practically every textbook, including the one I used.

I agree with the maths that you've written, but it's the same thing that the textbook explains, practically in the same way. The road to understanding is not the same for all people, I guess...

Sheldon Axler, Linear Algebra Done Right.

There are now videos! http://www.linear.axler.net/LADRvideos.html

BTW, Thank you for all these suggestions. Although I am not sure whether they make any difference for a programmer who is not yet fluent in linear algebra, I think they are great for people to explore once they are a bit more confident with the basic stuff.

HN discussions on previous 3 parts:

https://news.ycombinator.com/item?id=14480559 https://news.ycombinator.com/item?id=14497867 https://news.ycombinator.com/item?id=14549007

Reply on news.ycombinator.com