Hacker News Reader

> What are orthogonal vectors good for?

Any set of n linearly independent vectors B_a={\vec{a}_i}_{i=1..n} in a vector space can be used as a coordinate system, but the "computational cost" of finding the coordinates w.r.t. the basis B_a will be annoying. Each time you want to find the coordinate of a vector you have solve a system of linear equations.

A basis consisting of orthogonal vectors {\vec{e}_i}_{i=1..n} is way cooler because you can calculate the coefficients of any vector using the formula $v_i = (\vec{v} · \vec{e}_i)/||\vec{e}_i||²

Of course the best thing is to have an orthonormal basis B_s={\hat{e}_i}_{i=1..n}, so that the coefficients of a vector w.r.t B_s can be calculated simply v_i = \vec{v} · \hat{e}_i. A projection onto the \hat{e}_i subspace.

> Why would we ever want a basis besides the standard basis in real Euclidean space?

Hmm... I'm thinking eigenbases? The operations of a matrix A for vectors expressed in terms of its eigenbasis is just a scaling, i.e. {B_e}_[A]_{B_e} = Q^{-1} {B_s}_[A]_{B_s} Q = diagonal matrix = Λ.

> Is Euclidean space the only vector space out there? > Do vectors have to be lists of numbers?

Good point. If I bump this to 5-6 pages, I will try to include something about generalized vector spaces. Orthogonal polynomials could be a good one to cover.

maybe bump it to two different handouts?

> Each time you want to find the coordinate of a vector you have solve a system of linear equations.

This is wrong, unless I'm misreading you. The first time you want the coordinates of a vector in a nonstandard basis you need to construct the matrix that does the change of basis, which requires inverting a matrix (same cost as solving a LSE), but after that, it's just plain matrix-vector multiplication.

Computer graphics is full of fun topics that answer these questions in accessible and visible/tangible ways!

Skeletal character rigging in particular motivates a nice understanding of orthonormal bases and why a basis other than the identity matrix is very useful.

Even simple 2d conversions between screen space & pixel space, for example, can be useful motivational examples- its the thing your web browser did to render what you're reading. :)

Not that anyone would want to cover those topics in a 4 page linear algebra primer, but maybe there is some inspiration there for ways to include pictures that explain, instead of more math notation... ;)

The last 2 questions there are interesting and worthwhile topics for the interested student, but I'd say, just my $0.02, don't ruin a good thing by stuffing too much into it...

I knew the answers to these questions :) I'm just saying they belong in a discussion of the topics. Maybe I just like to take math slower than a 4-page summary.

This is my favorite introductory textbook on linear algebra which goes to great lengths to avoid matrix and determinant computations:

http://www.amazon.com/Linear-Algebra-Right-Undergraduate-Mat...

Thanks, i have been doing a lot of machine learning hacks lately and almost everything involves "vector spaces", 249 pages should be fun, just finishing up "Think Bayes" by Alan Downey.

I read Axler's book quite recently, after pouring through reviews of linear algebra books. I did like it quite a lot -- it has a very clean, approachable narrative style. Having completed it, though, I feel there's maybe not enough operational knowledge gained from it. Meaning, it explains the theory, the motivations, well, but one might augment it with Schaum's[1], or some such guided exercises.

[1]:http://www.amazon.com/Schaums-Outline-Algebra-Edition-Outlin...

Axler is not a text in applied linear algebra, it is a book you should read after you are already familiar with both basic abstract algebra and basic linear algebra (i.e. you know how to multiply matrices etc.), it then provides a nice motivation for linear algebra as a subfield of abstract algebra. You may still, however, not find the ideas of determinant, trace and other practical concepts very intuitive after reading Axler.

Basic abstract algebra? Definitely not needed to use that book to its full potential.

> Why would we ever want a basis besides the standard basis in real Euclidean space?

It think it can be explained from a SVD pointview, the standard basis may not likely be optimal one, and didn't capture the most variance. The basis with larger variance, will presumably contains more information.

After obtaining the good basis, we can approximate the original matrix using only the ones(often much smaller in number than standard basis) with larger variances. This is useful to find a shorter representation, yet preserves much of the distance information.

I think this post explain the intuition pretty clear:

http://www.cs.princeton.edu/picasso/mats/PCA-Tutorial-Intuit...

Wow, I just read that entire paper. Thank you for linking, it was super helpful!

Couldn't agree more! "Matrix is a function" is a super useful analogy, especially for us programmers, who deal with functions every day.

And matrix mult ~= function composition. I had a long ahaa moment reading this.

Well, it's not an analogy! There's a 1-1 correspondence between matrices and linear maps over a vector space with a fixed basis. "A matrix represents a function" is no more an analogy than "The symbols 'V', '5', and '五' represent the number five" is an analogy.

It's more like matrices are a useful way of representing certain functions, just like tally marks are a more useful way of representing numbers when you're counting the number of guests as they arrive but arabic numerals are more useful when you're doing arithmetic. Because these functions — linear maps — have a special structure to them, we can define matrix "multiplication" in a way that corresponds to the composition of linear maps.

In other words, it's not a happy accident. We invented matrices as a way to reason about and compute with linear maps more effectively. For example, "diagonal matrices" and "upper-triangular matrices" (among others) play important roles throughout linear algebra, but without the matrix representation of linear maps it'd be hard to talk about these things.

Here's the man you have to thank for putting all this together: http://en.wikipedia.org/wiki/Arthur_Cayley

That's a good idea. Perhaps when teaching people familiar with SVG, you can start by showing them the transform matrix syntax and explaining what it does. https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/t...

So... are you going to answer them?

Gilbert Strang explains clearly why we need change of basis with an example of image compression here, http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

How about "Is Euclidean space the only vector space out there?" And "Do vectors have to be lists of numbers?"

Perhaps with some examples?

http://en.wikipedia.org/wiki/Examples_of_vector_spaces

> "Do vectors have to be lists of numbers?"

No. They just have to satisfy the vectors axioms (greatly simplifying, you can add vectors together and multiply them by scalars , and they behave as you would expect - associative, commutative, distributive, etc as appropriate.)

So, in this regard, functions can be vectors, if you define addition to be pointwise addition. For example, consider the set of continuous functions on the unit interval [0, 1]. Add two together and you get another continuous function on the unit interval, multiply by a scalar and you get another continuous function on the unit interval, etc.

It turns out that, if you restrict to spaces of finite dimension, then Euclidean space is the only vector space out there, up to your choice of labelling the basis elements and up to a certain linear transformation.

In this sense, vectors can always be written as lists of numbers, but no, they don't have to be this in general. For example, you could have a vector space of smooth functions. It's infinite dimensional, and its elements (its vectors) are functions, not tuples of numbers.

You must also add the condition that your scalars are real numbers. There are many finite dimensional vector spaces that are nothing like Euclidean space: any finite extension of a finite field, for example. This comes from Galois theory, but is not just abstract nonsense. One application to computing is if your scalars are only 0 or 1 and all operations are done mod 2, then you have a framework for doing error correcting codes, among other things. We call this set of scalars either the finite field of size 2 or the Galois field of size 2, aka GF(2).

Yes, what I should have said is: if k is an algebraically closed field then n-dimensional vector spaces over k are isomorphic to k^n. Unless I've had too much to drink this evening.