Hacker News Reader

Comment by geokon | Hacker News Reader

Where is the text? Or is this an advertisement for a book?

Since this might get the right eyeballs here:

1. Does anyone have a good intuitive explanation for the meaning of the transpose operation? It seems to tie into a central symmetry between columns and rows (or equations and variables) that I'm having trouble articulating for myself

2. Why does every linear algebra book have a chapter on norms? It's always so incredibly boring and esoteric and I still haven't hit a situation where it's strictly necessary. The 2-norm seem adequate for most situations and comes out naturally from the least squares problem

PS: Strang is great. Meyer's "Matrix analysis and applied linear algebra" is even better (but more advanced)

View on HN

Check out Better Explained for a good intuitive guide to linear algebra (among other things): https://betterexplained.com/articles/matrix-multiplication/

Click on the numbered headings. They don't look like links, but take you to the text for that section.

Two interpretations of transpose:

- Let <-,-> denote an inner product. Then <Au, v> = <u, A^T v> is a definition of the transpose operation. Proof: (Au)^T v = u^T A^T v = <u, A^T v>

- The transpose maps a vector v to a linear functional (a kind of function) v^T, such that v^T (u) = <v, u>. Conversely, a linear functional of a vector can be transposes to a vector. In fact, it's the latter (linear functionals to vectors) that's more interesting. This fact generalises to Hilbert spaces as the Riesz representation theorem, where the consequences can be pretty amazing.

- An arbitrary linear operator L: H -> K tranposes to a mapping between their dual spaces L^* = K^* -> H^* by L*(f) = f o L.

- The outer product produces rank-one matrices, and all rank-one matrices can be expressed as outer products.

Oh yikes. So the rabbit holes goes quite deep

So the transpose is really ties to the whole idea of inner product operators/function? (I'm not sure what the exact terminology is there)

So thinking out loud a bit about the first interpretation... The inner to me represents an operator that maps vectors to a scalar that denotes the degree of "sameness" between two vectors. And it's a whole family of operators that all share certain properties. It just so happens that to write out this operator in matrix notation (for the standard inner product) we need to transpose/rotate the first vector which is yucky and why we prefer the bracket notation

It's a cool insight but it just makes it seem like a mechanical operator that flips a column into a row to make things work for one family of operators. I'm thinking more like in the case of an orthonormal matrix/basis. We know the transpose is it's inverse. It stems from the standard inner products of the columns being equal to one which is I find kinda interesting b/c the inner products suddenly become part of linear systems and not just a separate measurement of a degree of sameness.. but the headache for me is that I can picture an orthonormal basis, but I can't picture it's inverse! :)

PS: The orthonormal basis case and the transpose by extension further makes me feel that only the standard inner product seems to really map to something directly applicable in linear systems

PPS: The 3rd bullet is a bit beyond me. And the 4th is also very interesting but I'm not sure how to expand on that to build an intuition haha. I'll need to digest it further

For one example the notion of a generalized inner product and its induced norm is important for the derivation of conjugate gradient algorithm. My smarter friends have told me that anything involving cost functions and covariance matrices (as might arise in a log likelihood thing for a multivariate normal random variable) will probably end up with weighted norms as well, but I have likely mangled this. So to mangle things further I think that this is very important to many fields eg control or statistics

The coordinate-free concept behind transposes is the adjoint, which shows up in multiple places in mathematics. Basically, you have a formula:

<Ax, y> = <x, By>

where <,> is some inner product, and you say that the the linear transformation A is adjoint to B. If you take the usual inner product, then A is B^T.

A previous conversation: https://news.ycombinator.com/item?id=15948153

As for 2, norms (and general inner products) show up in more theoretical parts of math that use linear algebra. Maybe an applications-focused treatment doesn't need to focus so much on it?

> Where is the text?

Look for the word "Next" to find the button that takes you to the next page. Even though this is arguably the most important thing on the page, it is rendered in a very small font, and styled to not look like a link. Can't say I understand this design decision.

i mean, probably the sexiest application of linear algebra is machine learning and different exotic norms are very important there. sometimes you even have to learn a norm from data (metric learning).

at the very least i feel like it’s important to understand 0, 1, 2, and infinity norm.

Computer graphics is a pretty fun application as well.

Reply on news.ycombinator.com