Did not realize you could use MXNet declaratively (like Tensorflow/Theano) and imperatively (like Torch/Chainer). Can anyone speak more of their imperative usage of MXNet?
I learned about it last week, I don't seem to see too much benefit if the goal is good performance.
The computation graph is an in-memory datastructure that can be introspected by the program itself at runtime so as to do symbolic operations (e.g. compute the gradient of one node in the graph with respect to any ancestor input node).
theano implements this in pure Python and can generate C or CUDA code from string templates (in Python). tensorflow has to a Python API to assemble pre-built operators which are mainly written in C++ and use the Eigen linear algebra library.
But this sounds exactly like expression template.
Once the graph is defined, it can be passed along with concrete values for the input nodes to the runtime framework to execute the section of the graph of interest (possibly with code generation + compilation).