Billions of courses, web sites, job applications and HN posts. The subject seem to have taken off massively in the last two years. I mean image and speech recognition is pretty cool (when it works!), but hardly that earth shattering, is it?
The rate of advance using deep nets in signal processing will likely slow down now, but they aren't going away, not in the foreseeable future.
The hype around DNNs arose when we took our unbridled enthusiasm for what's they've achieved in DSP and extended it to other domains where data is less 'dense' and thus aren't as amenable to fast de/convolution in N-D space or time.
Will DNNs revolutionize or introduce all the techniques needed to achieve AGI/Strong AI? I very much doubt it. As yet, there's little sign that DNNs can perform relational operations on interdependent symbols, like the transforms available via type theory, bayesian nets, or predicate logic.
The multitude of disparate facts and semantics in a rich knowledgebase can't be organized into dense matrices the way that continuous signals can, so the SIMD operations that are so effective in DSP won't implement the rich transformations needed in a relational fact-based knowledge space equally as well, if at all. Thus DNNs almost surely aren't going to take us to the heights of logical or compositional thinking that human level intelligence requires.
But how far up relational mountain will DNNs take us? I suspect that won't be known for a decade or longer. But even if we don't reach the summit, we'll be considerably closer than we were before.
ML is a much broader field than just neural networks. The hype for ML, in general, I think is warranted. We hit an inflection point when AWS launched and scalable processing power became cheap. It became cheap to process tons of data and generate insights. I don't have hard numbers on this, but probably 90-95% of machine learning used in practice is NOT neural networks, and have accuracies in the 90%+ arena. So ML in general -- sure, hype warranted.
Neural networks are the new hot topic, and the hype isn't fully warranted yet. TensorFlow made them very popular in the developer community; this is a good thing because it's spurring more investment and research in ANNs. But for any given problem, odds are that a neural network is not the best (ie, most accurate or cheapest) way to solve it. Neural networks do have specific problem domains where they are the state of the art, but for most other problem domains there exists a better solution. So I'd say that neural networks are a little over-hyped right now, but with a new generation of developers learning about and experimenting with ANNs, that will change in a few years. I think we're about to see an explosion of ANN usefulness over the next few years.
TLDR: ML is very useful but is more than neural networks; neural networks need a little more progress to catch up to the tensorflow hype.
Here's a concrete example: Before spreadsheets existed, there used to be legions of accountants who created complex ledgers on paper and added up all numbers to track how a business was doing. You'd literally mail off your sales numbers to an accounting team somewhere and wait three days to get the latest report generated and sent back. Sure they had calculators to add numbers, but the computers of the day didn't understand how those numbers related to each other. The human still had to do most of the work to create the reports.
The big idea of spreadsheets was to make the computer manage the more complex task of knowing how different numbers in a report related to each other. It made most ledger tasks totally automatic once the initial report was defined. Now a single accountant could do the work of the entire accounting team - and more accurately and in less time! There were stories of the first spreadsheet testers having to delay mailing back their financial reports by a few days because their clients would be suspicious if they mailed them back too fast.
Nearly overnight accounting got a lot more efficient and companies made more money. T"What If" modeling that used to be too slow and cost prohibitive to do was now it was quick and easy. Companies could plan more intelligently. The spreadsheet was a true game changer.
This same pattern happens every time the bar is raised on the complexity of what can be automated and Machine Learning raises the bar one giant notch. Previously we were limited to automating tasks that a smart coder could describe as discrete steps in code. But with ML, the computer can figure out it's own rules just by looking at data. That means in many cases you can solve very hard problems just by collecting a lot of data. Lots and lots of things that used to be done by large groups of people will now be able to be done with a single computer.
In that sense, ML is a total game changer. Don't focus on the specific applications thus far. Focus on the idea that all kinds of tasks that used to require humans can now be automated with a little bit of applied ML. The opportunities are literally everywhere.
In a few years, ML won't be some esoteric technique used by a few people. It will be a core skill that everyone uses or touches in some way. It's going to creep into everything everywhere because it's just so darn useful.
The advances made by deep nets in signal processing will likely slow down now, but they aren't going away, not in the foreseeable future.
The hype around DNNs arose when we took our unbridled enthusiasm for what's they've achieved in DSP and extend it to other domains with data that's less 'dense' and thus aren't as amenable to de/convolution in N-D space or time.
Will DNNs revolutionize or introduce all the techniques needed to achieve AGI/Strong AI? I very much doubt it. As yet, there's little sign that DNNs can perform relational operations on interdependent symbols, like the transforms available via type theory, bayesian nets, or predicate logic.
The multitude of disparate facts and semantics in a rich knowledgebase can't be organized into dense matrices the way that continuous signals can, so the SIMD operations that are so effective in DSP won't implement the rich transformations needed in a relational fact-based knowledge space equally as well, if at all. Thus DNNs almost surely aren't going to take us to the heights of logical or compositional thinking that human level intelligence requires.
But how far up relational mountain will DNNs take us? I suspect that won't be known for a decade or longer. But even if we don't reach the summit, it'll be higher than we were before.