I'd find Rudin's argument a lot more convincing if she offered an existence proof, like using the same number of examples to train an equally discriminative SVM or random forest (or hybrid) that can equal the performance of AlexNet in 2012 on the ImageNet ILSVRC (or in another domain where DNNs are SOTA).

Until that can be done, I think few outside academia will invest time or money in alternative non-DNN methods in the hope of competing with today's even superior DNN variants. There's a decade of evidence now that DNNs are incontestable discriminators in numerous domains, relative to pre-2012 ML technology anyway.