Wow. As far as I know, this is the first time anyone reputable[a] has claimed to show (!) that the "manifold hypothesis" is the fundamental principle that makes deep learning work, as has long been believed:

  "In this work, we give a geometric view to
   understand deep learning: we show that the
   fundamental principle attributing to the
   success is the manifold structure in data,
   namely natural high dimensional data
   concentrates close to a low-dimensional
   manifold, deep learning learns the manifold
   and the probability distribution on it."

Moreover, the authors also claim to have come up with a way of measuring how hard it is for any deep neural net (of fixed size) to learn a parametric representation of a particular lower-dimensional manifold embedded in some higher-dimensional space:

  "We further introduce the concepts of rectified
   linear complexity for deep neural network
   measuring its learning capability, rectified
   linear complexity of an embedding manifold
   describing the difficulty to be learned. Then
   we show for any deep neural network with fixed
   architecture, there exists a manifold that
   cannot be learned by the network."

Finally, the authors also propose a novel way to control the probability distribution in the latent space. I'm curious to see how their method compares and relates to recent work, e.g., with discrete and continuous normalizing flows:

  "...we propose to apply optimal mass
   transportation theory to control the
   probability distribution in the latent space."

This is not going to be a light read...

--

[a] One of the authors, Shing-Tung Yau, is a Fields medalist: https://news.ycombinator.com/item?id=18987219