Read up on the calculus of variations if you want to be obscure with respect to neural networks. Genetic algorithms for hyper-parameter search is also interesting.

The "deep" part of DNNs has basically thrown mathematicians and statisticians into an infinite loop that they can't quite compute yet. It's a brand new world and we need them to participate.