Hacker News Reader

Comment by dreamcompiler | Hacker News Reader

Here's where this is confusing (to me at least).

The dimensionality of a vector space is the number of scalars needed to build a vector in that space. That's easy.

I also think of "rank" as synonymous with "dimensionality" but it's not. Or at least it implies a different connotation of the word "dimensionality." It's the number of dimensions in the notation rather than in the space. Or something. And now I'm off the rails. And this is before I start trying to think about a basis being "ordered pairs of vectors" or tensor fields or about how tensors transform.

View on HN

That's right. The dimensionality of the tensor space depends on both the rank of the tensor and the dimensionality of the underlying vector space. A rank-2 tensor based on 3-D physical space has 9 dimensions because there are 9 distinct pairs of the 3 underlying basis vectors. In general, a rank-N tensor on an M-dimensional space will have M^N dimensions.

Take a look at my definition of the "mathematician's tensor" in the top-level comments. The rank of a tensor is the number of row- and column-vectors you need to feed the tensor to get it to return a real number. AFAIK it's not related to 'rank' in the sense of the rank-nullity theorem.

To move from the mathematician's definition to the ML definition, pick a basis for your row and colum vectors. Now if you want the (i, j, ...) element of the multidimensional array, gather the vectors with one in the i-th position and zero elsewhere, the j-th position and zero elsewhere, etc. Then feed them to the tensor in that order.

It's easiest to see how it works with row vectors (rank (0,1)), column vectors (rank (1, 0)) and matrices (rank (1, 1)) and work from there.

There is in fact another definition of “tensor rank” which has everything to do with the rank of a matrix. For a tensor t in a tensor product of vector spaces VxW, define the rank of t to be the least number of summands possible in an expression t = v1xw1 + … + vnxwn.

If t is zero, then it rank is zero. If the tensor product is Vx(dual V), ie of type (n,m)=(1,1), then a tensor t can be considered as a matrix, and its tensor rank is the same thing as its matrix rank. You’re basically looking for the smallest way of writing the matrix as a sum of outer products of row and column vectors.

If you’re into quantum physics, then tensors of rank 0 or 1 are non-entangled, and tensors of rank 2 or more are entangled states.

Nice. Makes sense, thanks for the explanation

Reply on news.ycombinator.com