The article is intriguing, but is the objective of the mathematicians of more than theoretical value?
It seems to me that proving DNNs to be equivalent to SVM kernels won't improve them appreciably -- they won't become more accurate or precise, not faster, nor more economical. More importantly, the achilles heel of DNNs won't be fixed -- their inability to explain their reasoning or annotate their inner workings to enable tuning or repair.
Replacing black box linear algebra with black box math does not seem like much of a step forward.