Thanks, these posts are the only thing I've read that have made sense to me thus far (probably because they mostly confirm my suspicion, that there's not really a strong distinction between 'symbolic' and 'automatic' differentiation, but there are perceived differences based on whether the computation graph is compiled and optimized, and whether the graph is pretty-printed at some point...). Your Python symbolic differentiator is exactly a mini-Theano (or a mini-any deep learning framework). Each `Function` instance (or `Op` in Theano) defines its output and gradient with respect to its input (given gradient wrt output).