New Theory Cracks Open the Black Box of Deep Neural Networks | Hacker News Reader

New Theory Cracks Open the Black Box of Deep Neural Networks

wired.com

36 pointsbreck14 comments

NN look like black boxes because we’re not stripping out spurious axons in the network to understand the computational circuit. If you can remove a weight (w_ij -> O) from a NN with no change in the network’s function across a broad spectrum of inputs then it clearly wasn’t necessary in the first place as that weighted edge was not involved in the information processing. Training networks will not only tune certain weights to dial up on signal but they will also (effectively) zero out certain weights that don’t carry information (if they’re not effectively zeroed our then they introduce noise and compromise the network function), in effect reducing the noise and focusing on signal. For anyone interested I wrote a paper in Natures Molecular Systems Biology in 2008 which goes into this. It describes artificial gene regulatory networks but my background was in AI and the NN math is the same. I believe the results are universally applicable across a broad range of NN-class network. Lots of relevant remarks in the supplementary as well. This includes a description of an algorithm to prune the network of spurious interactions.

http://m.msb.embopress.org/content/4/1/213.abstract

Waste of a read! Reasoning that going from many input nodes to few output nodes results in compression (or shedding of nonessential data) isn't a brilliant theory, it's common sense.

The author even beats the dead horse of AI being a potential existential threat.

That's exactly what I was thinking. Wow, a neural network discards information at each layer by reducing the number of cells in each matrix? Water is wet.

How else would a network make a final decision? If it never discarded any information, it would be just as heterogeneous of a dataset as when it started. It would be as if a linear regression algorithm "fit" a dataset by connecting the dots, except even more useless.

I truly don't understand what this "new" theory is trying to say.

It's so basic I was thinking they could have called me, someone who studied neural networks only for half a year in uni, ten years ago. I could have told them that.

Either the journalist compressed the information a bit too much and lost the essential amazing insight. Or it's just not amazing.

Everything is obvious in hindsight. Too bad you didn't write up and publish your brilliant thoughts before this guy did.

The actual talk is pretty technical and sophisticated. The conclusion is not just that neural networks discard information; it's a rigorous derivation of exactly how they do so.

The insight may be to show why image recognition has turned out to be a fairly easy task for deep learning - because just discarding pixels (the right pixels, but still) gets you a very long way toward your solution. Other problems that a brain handles quite well may not yield to such a simple process/analysis.

Did you look at the actual research, or just this author's summary? I was also skeptical after reading this summary, but I found the presentation in the linked video to be far from trivial.

Indeed. I'm glad the grandparent saved me from reading the article, but I'm even gladder that you got me to watch the talk.

Yes, it might be common sense (to some). But the talk is a very lucid presentation, and also very technical. Definitely not a "water is wet" scenario.

So there’s not chance the brain does back propagation right? I wonder what it might do instead?

I wouldn't say not a chance, we have no clue what the brain does.

Only because a brain can learn building on old concepts doesn't mean anything about the basic algorithm being different. The brain is a much bigger and longer living neural network.

What they were explaining with the child learning character recognition with the knowledge about strokes. Why would you think our brain doesn't simply connect some of the stroke recognizing neural networks to a higher level neural network built for character recognition. You can increase the depth of the network basically indefinitely. Stack various neural networks to produce more complex ones, etc.

I don't think it's a good idea to be calling the human brain a neural network in this context, as cnns/dnns only sort of vaguely resemble brain neurons. The differences are massive, both due to different neurotransmitters as well as different connections being formed, and other stuff

That doesn't have much to do with my point, also the brain is the original neural network. If anything you should argue to call artificial neural networks by a different name.

How big of a deal is this from 1-10?

Like any other theory in science. Nuclear power or glow in the dark toys, usually both, if enough time pass

Reply on news.ycombinator.com