Yes we have engineered better NN implementations and have more compute power, and thus can solve a broader set of engineering problems with this tool, but is that it?
That's a lot of data to deal with, especially since you need to train it, running huge computations using each neuron.
I know nothing about hardware, and this is a very crude prediction/estimation of how AGI would happen, but my point is that we might be limited by Hardware for a few more years.
But that's not the case. Deep nets can model vastly more information / state than any other AI/ML method. Once Hinton (and others) showed how to train NNs with more than three layers (ca. 2006) it was finally possible to learn and store all that state. Then with the rise of GPGPUs soon after, deep nets became efficient as well. Thereafter several tasks that had been infeasible even using curated information became amenable to mostly brute force learning strategies driven only by labeled examples -- just lots of 'em.
The question now is how far can we extend DL's tools and examples. Are they sufficient to build higher level cognitive AI agents. Must AGI employ many thousands of deep nets? Or can all those specific-skill nets be folded together somehow into one unified "deep mind"?
Like you, I'm doubtful that today's very specific successes in DL will lead to higher level cognition in the foreseeable future. That path isn't at all clear to me.
Doesn't seem like he's trying to claim anything philosophical.