A convolutional neural network ought to have translational symmetry, which should lead to a generalized version of momentum. If I understood the article correctly the conserved quantity would be <gx, dx>, where dx is the finite difference gradient of x.

This gives a vector with dimensions equal to however many directions you can translate a layer in and which is conserved over all (convolutional) layers.