Can someone give a concrete example of the kind of theoretical properties they desire of ``new-style'' machine learning? The kinds of properties that ``old-style'' learning methods guaranteed?

People often complain about interpretability: in what sense is an SVM interpretable that a deep neural network is not?

Or is the worry about gradient descent not finding global optima? But why is the global optimum a satisfactory place to be, if the theory does not also provide a satisfactory connection between the space of models and underlying reality?

The arbiter of good theory is ultimately its ability to guide and explain practical phenomena. Which machine learning phenomena are currently most in need of theoretical elucidation?