Neural Networks making a come-back?
yaroslavvb.blogspot.com
yaroslavvb.blogspot.com
It is true that these things are becoming more popular. I've found in practice that a modern computer scientist is still more likely to solve a simple learning problem with some form of regression, if only because it's faster than training a NN.
I think NN is a broad enough category that no matter what you want to use or describe, you will have to qualify your "lets use blah" statements with a particular kind of neural network. Similar in spirit to statements like "lets use a parser" vs "lets use a LALR parser".
But back to the topic of new found interest on NNs, part of the reason is that there have been new developments in training algorithms which work significantly better than what were used traditionally. With these methods NNs require far less baby-sitting. NNs traditionally really required a huge lot of that.
The other reason is that sheer scale and size of the data sets that are available now, have forced machine learners to move from powerful but batch optimization algorithms (quadratic programming for instance) to simple and online gradient based algorithms that have been the forte of the NN community all along.
Training a NN is no different than regression. It is another name/technique for (some what systematically) creating a tower of increasingly complex regression functions. If the simplest(linear) one works, its imperative that one uses the simplest one in the interest of good predictive accuracy on unseen data. Bundled together with the low training time that parent mentioned, its a win win.
For vision based research I would suggest CVPR and ICCV conference and IEEE Pattern Analysis and Machine Intelligence journal.
I wonder if there is a lag between academia and the web?
A lot of the newest work is under a few different names: "deep learning", "convolutional deep networks", "unsupervised feature learning", etc.
Here's a great talk by Andrew Ng of Stanford, who's a recent convert to this area: http://www.youtube.com/watch?v=ZmNOAtZIgIk
(Note that this is quite a one-sided view of things, but it does convey the excitement of deep-learning researchers and the potential of what might be possible.)
Although I'm not in deep learning myself (I'm a computer vision researcher), here's a TL;DR as I understand it: rather than having people in specific domains such as computer vision or speech processing create their own features, the idea is to take raw inputs (pixels in the case of images) and train multi-layer neural net architectures that "learn" the relevant higher-level features in an unsupervised way (i.e., without labeled training data). Some of these seem to be pulling out interesting features and perform competitively on a few benchmarks in vision and other fields.
I'm not sold on this yet, because it seems like the complexity of designing features has merely been traded in for the complexity of designing different learning architectures, but it's certainly becoming quite popular these days (mostly led by Geoff Hinton of Toronto, Yoshua Bengio of Montreal, and Yann LeCunn of NYU).
For example, the Restricted Boltzmann Machine (http://en.wikipedia.org/wiki/Boltzmann_machine#Restricted_Bo...), as far as I understand it, seems to be a variation of neural networks.
If you can post a link to an article that covers recent work in the area and explains why they none of them are breakthroughs, I'd love to read it.
i'm not sure your caricature of the netflix winning solution is correct: i believe it was a blending of around 25 different models (including, i think, the RBM someone pointed about above) each in themselves quite varied from one another. this is typically how these challenges are won.
The netflix prize would not have been won when it was without rbm.
This one is related, but not the one I originally mentioned: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5197...
I was at the talk but can't for the life of me remember the title, sorry.