Greedy, Brittle, Opaque, and Shallow: The Downsides to Deep Learning

Or apparent ceiling. Sometimes ideas haven't really "ceilinged out" so to speak, but they've run into an artificial barrier due to available compute resources at a specific point in time, etc.

Somebody (Geoffrey Hinton, Andrew Ng, I don't remember exactly) made the point that neural networks were in a pretty moribund place, until the confluence of three things came along: more compute power, more data, and algorithmic improvements. Given that, one wonders what other older ideas are waiting to be resurrected by a similar confluence.

Agreed. Train a billion-parameter model on million-feature sensor inputs from "normal" conditions, and it will drive really well until its massive over-fitting runs into slightly unusual circumstances, like a tumbleweed rolling across the road. Then it will do something completely unpredictable, and people will probably die. ML plus pervasive surveillance can automate a lot of routine work, but it has a serious problem with outliers.

If you just take the visualization part we have a massive amount of work to do. Understanding how these networks solve problem in sophisticated ways will motivate new advances to mitigate shortcomings. It is a software problem that can have huge benefits. For example, see the paper Visualizing Loss Functions For Neural Networks as an example of the incredible insight to be gained with better visualization.

The intuition of loss functions hasn't changed much in the last several decades. If you have a local optimizer it needs to be able to find its way down further than it already is. Anything you can do to remove junk configurations from even being considered during optimization will help. But knowing when that is happening has been really hard because it's just so computationally intensive to generate plots like in that paper.

https://arxiv.org/abs/1712.09913

> But almost all the interesting problems in cognition aren’t classification problems at all.

That's BS. Knowing how to name or interpret event, to apply a symbol to the noise of sensory data, is a crucial part of cognition and it happens to be the foundation of the symbolic reasoning Gary Marcus and company tout so often.

The whole "emperor has no clothes" tone of these critiques is in itself false, because the people at the center of deep learning are well aware of its limits and trying hard to work around them.

I'll try to address the epithets one by one:

* Greedy - Yes, deep learning is data hungry. No one denies it. That is a great boon for storage companies and GPU makers, so we may briefly ponder the incentives of neighboring industries to push DL. However, there's a lot of work being done to diminish the number of examples an algorithm must train on before it can make an accurate prediction. You can Google "one-shot learning" and "zero-shot" learning for more info.

* Brittle - On the one hand, deep neural networks learn what they train on; on the other, they are pretty strictly judged by their ability to generalize to data they haven't seen before. In fact, it is this ability that has one them notoriety. For work on making them less brittle, see Hinton's recent work on "capsule networks" among other research. (Capsule networks are also deep learning.)

* Opaque - interpretability is a problem, but it's not as simple as calling DNNs a black box. (Yes, the billions of parameters are not really human readable, but they are machine-readable and we can apply functions to them to obtain insight into the model.) In fact, there are many approaches to interpretability. It's some of the most exciting research in AI and it's coming out of really strong labs.

Here's something recent from DeepMind: Learning explanatory rules from noisy data https://deepmind.com/blog/learning-explanatory-rules-noisy-d...

* Shallow - This is the hard one. Gary Marcus and others suggest that we can augment learning algorithms by pre-programming them with rules or knowledge of the world. Fine. But it would seem to me much more impressive that these algorithms arrive at knowledge without human intervention of the rules-based kind, which is precisely what Hinton and LeCun and Ng have advocated doing with unsupervised learning, which is cough also deep.

One of the weird things about the role that Gary has cast for himself as the gadfly of deep learning is that he's very critical of this one narrow branch of AI. In fact, the top labs using deep learning are already combining it with other types of machine learning algorithms. DeepMind's AlphaGo is DL + reinforcement learning + Markov decision process.[0] Pedro Domingo is very clear that combining algorithms across machine-learning and AI sub-disciplines is the most promising path.

I'm not sure I know of any "deep neural networkists" who espouse a dogma that would actually justify Marcus's strange obsession. Most people, including the godfathers of deep learning, are agnostic. They use algorithms that work and will combine them with other algorithms that work when those come along. It seems like a semantic quibble. The real work continues.

[0] https://deeplearning4j.org/deepreinforcementlearning

http://www.argmin.net/2017/12/05/kitchen-sinks/

To me, someone who doesn't have a second language strong enough to verify the accuracy of translation, I simply run things through google translate, to the target language and then run the output back to English. Its the translation equivalent of the game of telephone.

Take "the fat cat sat on the mat"):

"the big cat was sitting on the carpet" or "The cats of oil sat on the bed"

and lots of other things which are just odd..

Instead, we can acknowledge that Google Translate has reached a useful level of skill (enough to usually get the gist of an article), but not enough to be fully reliable, accurate translation. The Hofstadter article usefully demonstrates how far we have to go.

AlphaZero is a good counterexample. In contrast to AlphaGo, it has no priors.

It's pretty common to use DRL in real-world robotics e.g. in self-driving cars or robotic motion planing, so you don't need to simulate the real thing. The obvious issue with RL is that it needs even more iterations/episodes/epochs than a typical DL, making real-world development either impractical (time) or too costly.

Rocks, flatworms, and the Hoover Dam all have embodiment in the same complex world that we do, but none of them will ever coherently debate philosophy, because they have no structures capable of learning to do so. I’m not convinced that our RL agents do either.

That’s one possible solution, but it only works to the extent that your simulator accurately reflects reality. Any hard coded simulation is therefore doomed to fail badly, and any adaptable “simulation” isn’t really a simulation at all — just more neural architecture.

> So we need simulation both outside and inside agents.

We don’t even need a simulation outside of agents, if you just use physical robot bodies :) Humans do just fine learning without extra simulation environments — the real world is enough.

Of course, simulated environments are incredibly useful, but not in the ways you’re claiming. Mostly it’s because reinforcement learning is still so slow to learn that without a simulated environment it would take forever to test anything. Also, they allow repeatable, testable results that allow learning effectiveness to be measured scientifically, etc.

Also I know you can formulate RL such that you use a simulator as part of the learning algorithm, but IMO that is cheating at worse, and a misguided rewsearch path at best.

> Neural nets by themselves are not even the complete answer.

Only to the extent that human brains are also not the complete answer to intelligence (i.e. maybe there’s a possible brain architecutre much better than ours), but I don’t think that’s what you’re saying.

If you meant the current wave of artificial neurons are not the complete answer, then of course I agree. But I don’t think that’s what you’re saying either.

Show me the part of our brain that you think contains a perfect simulation of reality and I’ll concede. Now certainly, our brain can do a kind of “neural simulations” to understand and reason about consequences of actions, but that “simulation” (if we can even call it such) is learned.

[1] https://www.sciencedirect.com/science/article/pii/0003347265...

Edit: and patents, lots of patents.

Edit2: maybe this: https://github.com/gamalon/chimple2

I've been playing with the concept for a while however failing to get any good results. Debugging probabilistic programs is so damn long and difficult since bugs can show as subtle biases in output instead of clear cut deviations. (I described my approach here: https://www.quora.com/What-deep-learning-ideas-have-you-trie... ) For me, this is just a hobby.

Joshua B. Tenenbaum et al.'s group seem to name their approach program induction, I had called mine Bayesian Auto Programming. I see you are calling yours Bayesian Program Synthesis. Clearly we have similar intuition about the essence of the solution to AI.

I wish you better luck than me.

"Perhaps most notably of all, Geoff Hinton has been courageous enough to reconsider has own beliefs, revealing in an August interview with the news site Axios 16 that he is “deeply suspicious” of back-propagation, a key enabler of deep learning that he helped pioneer, because of his concern about its dependence on labeled data sets. Instead, he suggested (in Axios’ paraphrase) that “entirely new methods will probably have to be invented.” I share Hinton’s excitement in seeing what comes next.

I don't think so. Philosophers have theorized about abstract thought for thousands of years. The materialist metaphysics that is uncritically and unconsciously held by the triumphalist philistines isn't even capable IN PRINCIPLE of accounting for it.

In any case, AI is useful, and will continue to be, but the fanfare is just noise distracting from the truth of what it is.

To date, no work in DNNs has shown any potential for redressing problems of the common sense knowledge needed to know when your translated sentence is semantic nonsense. Likewise, no statistic-based learning method has shown itself capable of tapping into much less creating the deep knowledgebases and wealth of relations that give bare facts the semantic meaning needed to convey or understand subtleties in messages that are more than trivial.

Deft mimicry may win a Turing Test, but complex syntheses of thought (like writing a fictional story in which characters have internal thoughts and hidden motives that depend on relationships, subtexts, and dependencies) — no form of AI has yet shown any potential to solve such problems. Too much internal state and complex relations must be modeled, much less, the ability to extend and translate these to many possible worlds — as humans do every day.

No, statistics-based AI techniques (like deep nets) show little promise to grow indefinitely unto full (wo)manhood.

>> There's no evidence for this, and I think it underestimates the large and well-funded machine learning community.

Well- deep learning has been around for a few decades now. LeCun's LeNet-5 is already 20 years old. LSTM is 21 (from the Hochreiter and Schmidhuber paper). Those are the two best-known today architectures, but work on deep nets goes way back to the '80s.

So those are old algorithms and techniques and they've had the same limitations for 20+ years. What makes them "likely to be solved soon" now?

" In 2006, a publication by Geoff Hinton, Osindero and Teh[26][27] showed how a many-layered feedforward neural network could be effectively pre-trained one layer at a time, treating each layer in turn as an unsupervised restricted Boltzmann machine, then fine-tuning it using supervised backpropagation.[28] The paper referred to learning for deep belief nets."

Perhaps this usage ("deep belief networks") was too effective, as so many marketing terms prove to be. Hinton is likely well-aware of the importance of terminology that captures the imagination of the public (and the marketplace for research).