Comment by struct | Hacker News Reader

Comment by struct | Hacker News Reader

Not to sound overly critical, but I don't enjoy the tone of this paper: presenting the concepts (e.g. graph operations) and features (e.g. automatic differentiation) of TensorFlow as new and novel when in fact very similar systems like Theano have existed since at least 2010 (which they finally get round to mentioning 14 pages in). They really should have shorn the paper in half and just focused on the distributed bit, which is the really novel and exciting bit.

Addendum/Edit: With that said, I don't want to disparage the amazing technical achievement of the Google Brain team and the way TensorFlow really smartly reuses and improves on the concepts that make Theano really powerful. I'm following the project very closely and TensorFlow has a very high chance of being the foundation for my future projects :)

Not to sound overly critical ;), but the word "novel" doesn't appear in the paper and the word "new" only appears in tehnical contexts like "quickly experimenting with new models".

Where does it say that its automatic differentiation feature is new and novel? It actually calls out Theano in the paper:

"Like Theano and Chainer, TensorFlow supports symbolic differentiation, thus making it easier to define and work with gradient based optimization algorithms."

EDIT: Okay, I see that you already saw the reference. I guess I didn't find the tone as bad as you did. :-/

It's confusing but automatic differentiation, symbolic differentiation, and finite differences are all different approaches. "Automatic differentiation" typically refers to reverse order chain rule. Higher level libraries do really neat stuff with it. For example autograd can differentiate much of your numpy calls, even of composed of many complex functions.

Even on distribution, I fail to understand where the novel idea is. I must miss something.

Sadly, most are only even paying attention because of the 100kg gorilla in the room…

Reply on news.ycombinator.com