It doesn't do well - e.g. on the toy example of MNIST they report 98.96% which is, frankly, rather horrible.

They claim that deep neural approaches get 98.75% or 99.05% by referencing obsolete decade old results, while in fact state of art exceeds 99.8% (i.e. 0.2% error rate, which is five times lower than 1.0% error rate reported in this paper). I have seen MNIST given as a homework exercise for undergraduate students in ML/NN class, and getting 99.0% would indicate that your code has some serious bugs, a decent undergrad with no prior experience can get 99.7% on MNIST after a few lecture introduction to basics and a dozen hours of homework coding practice.

It might be an alternative for deep neural networks, but I doubt it will be an alternative for deep convolutional neural networks. The key breakthrough for deep learning was the stacking of many convolutional layers and being able to train those (key elements include SGD for large scale, dropout or batch norm, etc.). Perhaps if they build some convolutional features for the decision tree they will be able to achieve comparable results, but it will start to look a lot like a deep convolutional neural net (i.e., the same thing with a different name).

In the conclusion section the authors said that :

"If we had stronger computational facilities, we would like to try big data and deeper forest, which is left for future work." and that:

"As a seminar study, we have only explored a little in this direction."

That's exactly what I was going to post, but you beat me. I found surprising that they claim competitive performance over neural networks just using a CPU. The advantage I think neural networks have over all other methods is how efficiently they run on GPUs. I would love to see other methods take advantage of modern hardware or parallelism and see if they can catch up with neural networks on image classification.