Hacker News Reader

It's not true, though. A list of numbers in R is not a vector until you imbue the position of each of those numbers with a geometric basis, or, alternatively, relate that list of numbers to the set of all lists of such numbers along with some algebraic operations which turn it into a vector space.

This is similar with tensors. It's even harder to talk about them, though, without the geometric/algebraic bits.

They use tensors here to mean simply multidimensional array. And as this discussion shows, that choice of words is unfortunate and confusing.

What I'm seeing is that physicists and geometers disagree with programmers about what "tensor" means -- not in that their views are inconsistent, but in that the programmers' view is too limited for the things that the physicists and geometers want to do with tensors.

They also disagree about what "vector" and "matrix" mean for the same reason, but they can't really put that horse back in the barn. But "tensors" remain unfamiliar enough that this disagreement can come up whenever anyone says "tensor".

Here's a possible truce:

- Programmers should default to talking about "arrays"

- Geometers and physicists should default to talking about "vectors", "matrices", and "tensors"

- We all generally recognize that some (vectors|matrices|tensors) are arrays, and some arrays are (vectors|matrices|tensors), and it's not the worst thing ever if someone uses one to mean the other

- We all generally recognize that you can sometimes do damn cool things by using an array as a tensor or a tensor as an array, so there isn't that much benefit in shouting "NO THEY'RE TOTALLY DIFFERENT".

Eh, I think as you stack more and more "structure" atop something it becomes less and less appropriate to confuse it with its representation. There is afterall already a completely appropriate, cross-disciplinary term for what "programmers" here call "tensors": multidimensional arrays, or even nd-arrays.

No mathematician will bat an eye if you say that a tensor can be represented by an nd-array---but if you equate them, then you're clearly missing something.

So... Do "programmers" think of "matrix" and "tensor" as "synonymous"?

edit: that was actually a serious question, because i really don't understand this idea that programmers have a common notion of what a matrix is.

I didn't mention the term "matrix". I think it has a much more complex story with respect to common use by "programmers" along with vector. Since linear algebra is such a commonly taught subject there are lots of people with practical working knowledge of matrices. If you don't have the geometric or abstract algebraic perspective here then matrix is almost certainly "2d-array" at first blush, but you'll also admit that there's a sort of non-trivial relationship between these "matrices" and size-conformant "vectors" (1d-arrays) and another one between two size-conformant matrices. This distinction is made from time to time [0][1].

Finally, of course, a matrix is a special case of a tensor. Tensor is such a broader topic that it becomes more and more difficult to talk about it without its geometric underpinnings. This is what I was writing about above.

[0] Is a black-and-white image a matrix? I'd imagine a lot of people would have a bit of a tough time saying yes directly---and for good reason. But there's also a reasonable argument for pretending like it is, and even reasonable arguments for turning it into one!

[1] Numpy distinguishes between nd-arrays and matrices exactly right in that matrices are 2d-arrays only and are imbued with multiplication as composition of maps. In practice, many people I know who are Numpy users just think of matrix as a convenient way to overload *, though.

poor choices of name happen all the time. we can't change them after they set in, so we have some really dumb use of terms today in the context of 5 or 10 years ago. it even happens inside of software development alone - the use of "responsive" in web design, which always used to mean "responds quickly", not "adapts to resolution and aspect ratio".

std::vector was an exceptionally poor choice of name...

is it so hard to just call it pseudotensor like it is? its confusing for people trying to research the subject when they come across all kinds of geometry and calculus stuff which is irrelevant to the usage here.

The use of "functor" in C++ is pretty terrible, too.

I agree it's unfortunate.

I think this is really a flimsy use of a technical term which does have a precise meaning in the context of mathematical physics to make the idea of multi-dimensional array seem "deeper" than it really is.

There are enough hard problems in data science - I don't think it needs to burdened with terms like this.

Especially if information geometry ever grows and data scientists start to actually have to use real tensors.

A 1D list of numbers is not necessarily a "vector" in the Differential Geometry sense. The definition of a vector or tensor is based on how it transforms as the co-ordinate system transforms. See e.g. http://en.wikipedia.org/wiki/Covariance_and_contravariance_o...

One weird consequence of this is that if A and B are vectors, the cross product (AxB) is NOT a vector in the tensorial sense. [Aside: Cross products are really weird since they are defined in the familiar way 3 and 7 dimensions only! (see http://math.stackexchange.com/questions/185991/is-the-vector...]

The word "vector" just means an element of a vector space: a collection of "things" that you can add together and multiply by numbers. The vector space R^n is defined as the collection of all lists of n real numbers, so you are right in that sense.

But usually when people say that a vector "is" a list of numbers, they mean that if you choose a coordinate system, then the coordinates of a vector uniquely identify it. This is exactly analogous to locations on the earth being uniquely identified by their latitude and longitude: we first have to make a (somewhat arbitrary) choice of coordinates. You wouldn't say that a point on the earth "is" a pair of numbers, though. The case of R^n is confusing, because if you choose the usual coordinate system, then the coordinates of a vector are the same list of numbers as the vector itself.

I like to think of tensors as functions that map some number of vectors to some other number of vectors. But you can add together two tensors (with the same number of inputs and outputs) or multiply a tensor by a number, so you can also think of them as vectors in some other vector space.

Almost seems like tensors are natural transformations in the category theoretic sense.