Just gave a brief run through the paper, and I'm curious: is reverse-mode differentiation in this context similar in concept to adjoint-type methods used in computational fluid dynamics (and increasingly, in design optimization in other fields)?

Backwards-propagation of a gradient calculation to allow for extremely high-dimensional parameter spaces is a trick that has been around for awhile in some circles, but also seems to have been missed in a lot of other disciplines. I'm seeing a lot of publications making use of it in the last few years, and it's pretty exciting to see them used in more places. My hope is that more scientific and engineering analysis codes will expose derivative interfaces for use in numerical optimization.

Here's some short papers with some background for those interested in the topic from a physical engineering perspective:

http://www.piercelab.caltech.edu/assets/papers/ftc00.pdf

http://www.nt.ntnu.no/users/skoge/prost/proceedings/npcw09/A...