Hacker News Reader

"The most striking difference between Tensorflow and other numerical computation libraries such as numpy is that operations in Tensorflow are symbolic. This is a powerful concept that allows Tensorflow to do all sort of things (e.g. automatic differentiation)"

Sorry if this is a stupid question, but can someone explain how symbolic operations allow automatic differentiation or link me to a good explanation?

View on HN

You can easily differentiate a set of (computer defined) operations formed of composition of standard operations (sums, products, etc), as in, you can define another set of computer operations that is the exact, precise derivative of the first. Wikipedia is a bit dense, but it's explained here: https://en.wikipedia.org/wiki/Automatic_differentiation

"Symbolic operations" is a fancy way of saying you build up the full mathematical equations with operator overloading instead of computing the results at each individual steps. Having the full equations then allows to differentiate with regards to a variable of your choice included in that equation

Something like lazily calculating/evaluating an operation, right?

Yep (from a compilation perspective)

That makes sense. Thank you!

Reply on news.ycombinator.com