AI Researcher wants computer scientists to move beyond the matrix | Hacker News Reader

AI Researcher wants computer scientists to move beyond the matrix

quantamagazine.org

12 pointsamrrs13 comments

This was underwhelming. A whole load of intro fluff about this person's upbringing and then it turns out she just wants to use tensors, as if that's some radical new idea.

If you're not familiar with tensors, where a matrix is a two-dimensional array of numbers, a tensor is an n-dimensional array.

Why this idea is not radical: ever hear of Tensorflow (one of the most popular AI libraries)? Or Google Tensor, which is a TPU (tensor processing unit)? Or Nvidia Tensor cores?

Computer scientists are already well aware of tensors and make extensive use of them, when it makes sense to do so.

I think TPUs and Tensorflow misname multidimension arrays as tensors and don;t actually use tensors. It could be the case I'm being a stickler https://math.stackexchange.com/questions/1134809/are-there-a...

There's a certain amount of ambiguity about the term "tensor" and what it means in any given context. I think there was a fairly highly discussed article or essay here on HN a couple of years ago where readers seemed to realize people were using the term in similar, but not identical ways.

How would you represent a tensor in code, if not by using a multidimensional array?

Ok, you could use a flat array and index into it depending on the dimensions of the tensor (as is often done for matrices), but that's really just a way of saving ram or cpu cycles rather than a different representation.

The point is that just because something can be represented as the result of malloc(), doesn't make the data structure the same as that thing. That is, what is:

float* x = malloc(width*height*channels*sizeof(float))

Is x an "Image"? Is x a "tensor"? Is x a "raster"?

For tensors specifically, if x is not itself the product of vector spaces, then it's not a tensor.

tl;dr: While every rank-N tensor can be represented with multidimensional arrays, not all multidimensional arrays are rank-N tensors.

They are isomorphic. That's as close as equal you're ever gonna get - it's turtles all the way down.

In physics we say that an object is a tensor if it transforms like a tensor. Self-referential, yes, but hints at the fact that not all possible multidimensional arrays are tensors.

This is talking about tensors in AI.

I don't know much about tensors used in AI or in physics. But I do know quite a bit about mathematical constructs being used across different domains and very often the definition is subtly different, so you cannot make a statement about the definition of tensors in physics and assume it'll hold for tensors in AI.

I think the original title, “The AI Researcher Giving Her Field Its Bitter Medicine” represents the article better. Not terribly clear the purpose of this article. It looks like some sort of wokeness promotion in Computer Science. I remember Anima Anandkumar was going after Yann LeCun pretty hard on Twitter, when that story was circulating. Oh yeah, and “use tensors”.

It's too bad this article never links to the paper [1] being hinted at.

In late 2012 (first submission) tensor decomposition was likely very interesting (the paper has >1000 citations!). But given that this article doesn't distinguish between "use tensors" and "use tensor decomposition, not SVD or similar for old school setups" I think the article is super confusing.

Perhaps the author didn't get the distinction, and decided to write up a "renegade/rebel" piece that is sadly a non sequitur.

[1] https://arxiv.org/abs/1210.7559

The moment I read the heading, I sort of guessed this was about Anima Anandkumar. She is making good progress in NVIDIA as version 2 of herself. Her previous persona was controversial and polarising. Good to see the positive change IMHO

Here I was hoping it would be an admonition to use tools like heuristic inference rather than just making ever-denser neural networks.

tldr: use tensors over matrices

Reply on news.ycombinator.com