Show HN: Hottbox – Higher-Order Tensors Toolbox | Hacker News Reader

Show HN: Hottbox – Higher-Order Tensors Toolbox

github.com

37 pointsIlyaKisil20 comments

Being a physicist in computational (general) relativity, when I read about a "tensor toolbox" I would think about a code implementing some kind of tensor algebra, such as a syntax to do tensor contractions.

Sorry for misleading. I am aware that the term "tensor" was originally introduced within physics. But then it was adopted in chemometrics, signal processing and machine learning in a context of N-dimensional arrays of data. As such, this toolbox is focused on multi-way analysis and tensor decompositions of N-dimensional data arrays (tensors). However, fundamental operations include folding/unfolding of the data, tensor-matrix product and contration with a vector or another tensor. The latter is not implemented at the moment, but will be in future releases. Not sure whether these opertations carry the same meaning as in physics though.

I don’t want to jump on top of a pedant pile, because I think it’s really cool that you’re out there making useful stuff.

The group where I did my PhD was a Numerical Relativity group and I now work in Machine Learning, so I can appreciate where you’re coming from.

However, a Tensor has a very precise mathematical meaning, and has done for centuries (dating back to at least Voigt, and arguably as far as Gauss). Even in machine learning, people recognise that they are abusing the term tensor by restricting its use to Tensors expressed in the canonical orthogonal basis of E^n.

I really think we should be discouraging this debasement of our mathematical terminology. It’s just not helpful at all.

I do agree that tensors are much more then just an array of data with N-indices and that numerical methods are oftentime forget about that.

I am looking for a python library that handles tensors, in the precise mathematical sense. Can you recommend one please?

Maybe https://cadabra.science/ ? I haven't used it myself, but it's linked from the main page of http://www.sympy.org/ , which would also allow you to define tensor operations yourself if necessary.

Do you have any recommended readings for ML people from a CS background wishing to get an introduction to Tensor analysis?

I would suggest to search on YouTube for Tamara Kolda: Tensor decompositions. She also published a very good introductory paper “Tensor decompositions and applications” in 2008.

That is a great talk. Thanks for the link

What "higher-order" part means here? What is the difference to regular tensors?..

Basically, order of a tensor is the number of dimensions of an array of data. Vector - one dimensional array or a tensor of order 1 Matrix - two dimensional array of data or a tensor of order 2 Three and more dimensional arrays of data are tensors of higher order.

As was stated by others before, I find the name very misleading. I know "Higher Order Tensors Toolbox" sounds way cooler than "ndarray toolbox", but wouldn't the second one be closer to the truth?

Furthermore, if 1-D Arrays are called vectors, and 2-D Arrays are called Matrices and 3-D (or higher) Arrays are called Tensors, does that not automatically mean that every Tensor is higher order? ...

This isn't quite right, but it's a common misconception. Any rank-2 tensor can be represented in matrix form, but not all matrices are tensors, similarly with rank-1 tensors and vectors.

The distinction is important because thinking about the way you have presented leads to confusion about what tensors are...

Sorry this is just wrong. Maybe you’re trying to get at the “co/contravariant” properties of tensors, in which case your statement can be more clearly stated as, e.g., the space rank (2, 0), and rank (1,1) vectors, admit different interpretations as internal hom spaces of vector spaces. But in any interpretation of your statement the distinction is never important because all spaces distinguished by this distinction are isomorphic via cononical isomorphisms.

You might want to think that through a bit more, leaving aside the issue of needing the underlying vector space, are you sure you are comfortable with the statement that all matrices are tensors?

Yes absolutely. Given a matrix as an array of numbers there's a number of natural ways to interpret it as a tensor. What are you uncomfortable about?

Was just explaining the meaning of “higher-order” within the context of this library. But thanks, now I have a clearer picture of what to include in the main README file and notebooks with tutorials, so to avoid misleading.

Can you explain this distinction in more detail?

Sure! There are several different ways to define them, but fundamentally tensors ar geometric objects that define linear relationships with respect to particular vector spaces. They are often defined somewhat loosly by their behaviour under transformations.

In context we often refer to scalars, vectors,matrices as order/rank 0,1,2 tensors (higher order tensors don’t have the same sort of common shorthand). This works fine when you have the context of the underlying vector space, and nderstand the “rules”. Physicists do this a lot, and they often love shortcuts :)

However, there is a growing use/abuse of the terminology (see machine learning) to just mean n-dimensional arrays. The analogy is drawn that a matrix is a 2d version but you can have 3D, 4d, etc. While it’s true that a NxN matrix can represent a tensor (given the context as previously) that misses most of the structure... as such, it’s an unfortunate use of the name.

Interesting choice of name? ;)

Reply on news.ycombinator.com