Yes, I've seen what neural nets are now capable of- they are capable of
exactly what they were always capable of, except "now" (in the last few years)
we have more data and more compute to train them to actually do it. Says Geoff
Hinton [1].
I have also seen what neural nets are incapable of. Specifically,
generalisation and reasoning. Says François Chollet of Keras [2].
AI, i.e. the sub-field of computer science research that is called "AI" and
that consists of conferences such as AAAI, IJCAI, NeurIPS, etc, and assorted
journals, cannot progress on the back of a couple of neural net architectures
incapable of generalisation and reasoning. We had reasoning down pat in the
'80s. Eventually, the hype cycle will end, the Next Big Thing™ will come
around and the hype cycle will start all over again. It's the nature of
revolutions, see?
So hold your horses. Deep learning is much more useful for AI researchers who
want to publish a paper in one of the big AI conferences, and to the FANG
companies who have huge data and compute, than it is to anyone else. Anyone
else who wants to do AI will need to wait their turn and hope something else
comes around that has reasonable requirements to use, and scales well. Just as
the original article suggests.
_________________
[1] http://techjaw.com/2015/06/07/geoffrey-hinton-deep-learning-...
Geoffrey Hinton: I think it’s mainly because of the amount of computation
and the amount of data now around but it’s also partly because there have
been some technical improvements in the algorithms. Particularly in the
algorithms for doing unsupervised learning where you’re not told what the
right answer is but the main thing is the computation and the amount of
data.
[2]
https://blog.keras.io/the-limitations-of-deep-learning.html Say, for instance, that you could assemble a dataset of hundreds of
thousands—even millions—of English language descriptions of the features of
a software product, as written by a product manager, as well as the
corresponding source code developed by a team of engineers to meet these
requirements. Even with this data, you could not train a deep learning model
to simply read a product description and generate the appropriate codebase.
That's just one example among many. In general, anything that requires
reasoning—like programming, or applying the scientific method—long-term
planning, and algorithmic-like data manipulation, is out of reach for deep
learning models, no matter how much data you throw at them. Even learning a
sorting algorithm with a deep neural network is tremendously difficult.