Those are all real problems, and they are also real problems that everyone who works with AI knows about. This type of article is a nice corrective to the hype, but it faces a similar risk of being too one-sided: in this case too critical. In a way it's a shallow criticism of shallow hype, its necessary straw man and prelude.
> But almost all the interesting problems in cognition aren’t classification problems at all.
That's BS. Knowing how to name or interpret event, to apply a symbol to the noise of sensory data, is a crucial part of cognition and it happens to be the foundation of the symbolic reasoning Gary Marcus and company tout so often.
The whole "emperor has no clothes" tone of these critiques is in itself false, because the people at the center of deep learning are well aware of its limits and trying hard to work around them.
I'll try to address the epithets one by one:
* Greedy - Yes, deep learning is data hungry. No one denies it. That is a great boon for storage companies and GPU makers, so we may briefly ponder the incentives of neighboring industries to push DL. However, there's a lot of work being done to diminish the number of examples an algorithm must train on before it can make an accurate prediction. You can Google "one-shot learning" and "zero-shot" learning for more info.
* Brittle - On the one hand, deep neural networks learn what they train on; on the other, they are pretty strictly judged by their ability to generalize to data they haven't seen before. In fact, it is this ability that has one them notoriety. For work on making them less brittle, see Hinton's recent work on "capsule networks" among other research. (Capsule networks are also deep learning.)
* Opaque - interpretability is a problem, but it's not as simple as calling DNNs a black box. (Yes, the billions of parameters are not really human readable, but they are machine-readable and we can apply functions to them to obtain insight into the model.) In fact, there are many approaches to interpretability. It's some of the most exciting research in AI and it's coming out of really strong labs.
Here's something recent from DeepMind:
Learning explanatory rules from noisy data
https://deepmind.com/blog/learning-explanatory-rules-noisy-d...
* Shallow - This is the hard one. Gary Marcus and others suggest that we can augment learning algorithms by pre-programming them with rules or knowledge of the world. Fine. But it would seem to me much more impressive that these algorithms arrive at knowledge without human intervention of the rules-based kind, which is precisely what Hinton and LeCun and Ng have advocated doing with unsupervised learning, which is cough also deep.
One of the weird things about the role that Gary has cast for himself as the gadfly of deep learning is that he's very critical of this one narrow branch of AI. In fact, the top labs using deep learning are already combining it with other types of machine learning algorithms. DeepMind's AlphaGo is DL + reinforcement learning + Markov decision process.[0] Pedro Domingo is very clear that combining algorithms across machine-learning and AI sub-disciplines is the most promising path.
I'm not sure I know of any "deep neural networkists" who espouse a dogma that would actually justify Marcus's strange obsession. Most people, including the godfathers of deep learning, are agnostic. They use algorithms that work and will combine them with other algorithms that work when those come along. It seems like a semantic quibble. The real work continues.
[0] https://deeplearning4j.org/deepreinforcementlearning