I'm not an AI researcher (although from my internet reading that's not a hard title to claim ;)) but I feel that ML/DL can't go much farther than they already have. The concept of "we just need more power" is an obvious fallacy to me.
I mean, all these "structures" and algorithms that people create in software, each a little different than the last, each with different performance/result trade offs. But nobody actually understands the fundamentals as to what is behind some of the very impressive results we are seeing (due to the technology allowing us to throw computing horsepower at it).
I'm not a researcher either, so it's difficult for me to articulate exactly what problems I see, but (shameless self plug) I did write an article a little while ago attempting to: https://medium.com/@danShumway/modern-ai-techniques-arent-wo...
I tend to be fairly dismissive of inference because what you end up with is a highly specialized algorithm rather than something that can easily continue to adapt. I suspect inference would probably fall into Jordan's category of "things that we call AI but probably shouldn't."
But that's not to dismiss how important fast/cheap inference has been in allowing companies to actually build things with AI.
Of course this is defending a strawman. I don't know anyone that said it was just a computing power issue. In fact I think most people vastly underestimate the role of computing power. Even algorithmic improvements are enabled by computers letting researchers do experiments that would have been impossible before. And by doing lots of experiments they gain intuition about the problem, that they wouldn't develop in a vacuum.
In 50 years people will laugh about how poorly these things were named. Just like we now laugh about what symbol we chose for a source of electrons in a circuit : "+". Whoops.
the little problem here is that it took 4 billion years of computing and a computer the size of a planet to come up with the nifty machines that are our brains. So unless you have brought a lot of tea and biscuits I think we should really think twice if just throwing more training and power at overly generalised algorithms is a good and realistic path forward.
It's not so much that the idea of "throw more things at it" is impossible, it's that it's a questionable path towards human intelligence. If you just want human intelligence without any further understanding of how to make it there are cheaper ways already
That is not impossible to simulate, even if it includes the entire lineage of the humans. Also, nature prefers generalized algorithms as well, starting from DNA.
1. We don't know all the mechanisms that a brain employs to achieve intelligence. We see billions of interconnected neurons and we assumes "Yea, this might be generating intelligence".
2. We don't know if we are already at some fundamental limits of intelligence. For example, you can see may instances in nature where a pattern emerged that maximizes some sort of efficiency. (Like Honeycomb pattern). So, the end result of this will be that, even if we transfer the process by which our intelligence work to a machine, it will have the same performance as an average human brain...
Machines are the tools we make to aid the above.
Maybe throwing more power at the current solutions won't ever make the progress we want, we need to find our car, so to speak. And a lot of people are working on that.
I believe 3 orders of magnitude more processing power we would achieve amazing results in a decade; not AGI type of results but very close to it from our perspective.
Part of the reason is that I now believe you can simply "bruteforce" some problems with existing ML algorithms (like thousands of layers deep neural nets) but more importanly, one(not me) could test new ML algorithms that are not feasible now (I don't have any examples) and people would be able to itterate much faster in developing such algorithms.
At current trajectory, we're not headed towards a general intelligence. Progress has been made, but there are big gaps. Smart home devices are a great case in point. They are somewhat flexible in the voice commands they accept. Specific phrasing and pronunciation are not necessarily required. Their responses and speech, however, are all pre-programmed and templated by humans.
Edit: There is potential for more breakthroughs in the future, but I am not seeing them on the horizon at the moment.
* Wavenet, now productionized at Google as text to speech
* Alpha[Go]Zero
* Neural Machine Translation, on production at Google
For more perspective: https://arxiv.org/abs/1801.00631