I'm a deep learning researcher so I can speak to this a little. Until roughly 2010 neural networks were barely used and broadly considered to be less efficient and less accurate than rival techniques (decision trees, support vector machines, etc.). For decades, far more work was put into these other approaches but the amazing thing about neural networks is that, even in a relatively primitive and neglected state, they outperformed traditional techniques. AlexNet is the poster-child of this phenomenon, but look at facial recognition, speech recognition, etc. These are all fields where older approaches are just dead. Nobody is working on SVMs for facial identification or object pose estimation anymore.

It's more accurate to say that neural networks succeeded against the research darlings of the time despite being relatively ignored for decades.