The state of the art has become a lot better in terms of training deep (multi-layer) neural networks. The naive approach (start at a "random" weight setting, use gradient descent) that works on a convex error surface fails catastrophically on deep, complex neural nets. (Shallow neural net training is a non-convex problem as well, but seems to be "less non-convex" in practice.) In the past 10 years, people have become better at getting past that.

Theoretically, only one hidden layer is necessary for neural nets to be universal. Thus, for a long time, most research focused on single-layer networks because those were "good enough" to model any mathematical function. The problem is that convergence, for single-layer nets, can be very slow (especially given that you're often doing stochastic gradient descent when working large data sets). Single-layer nets are often very difficult to audit. There's a lot of "cross talk" where unrelated features are mapped to the same "space" in the network. So you have a "black box" that is hard to interpret.

Deep neural nets (which is what ML researchers call "deep learning", until MBAs start abusing the latter term, as they have with "big data") have come back into style over the past few years, due to recent research into how to make them actually perform, and findings about superior convergence with the right conditions. In deep nets, there's often a problem of signals either amplifying or vanishing as they propagate throughout the net. The former leads to saturation (the neural net moves very slowly from a suboptimal but flat place on the error surface) and the latter is too linear and unlikely to pick up interesting features. Even now, making deep neural nets not sensitive to initial starting conditions is an unsolved problem, but there's been a lot of progress.

I would hazard the guess that the convolutional technique is a lot more useful in deep neural networks than it is in single-hidden-layer neural nets.