Comment by lrei | Hacker News Reader

Comment by lrei | Hacker News Reader

Parent Full thread

I've heard this argument ever since Norvig's Unreasonable Effectiveness of Data. While having a ton of data available is great, it has its limits. I believe you are overestimating the effectiveness of data (as, imo, Norvig did). And here specifically, it's not the case for the hype:

from the NYT article [1]: "The achievement was particularly impressive because the team decided to enter the contest at the last minute and designed its software with no specific knowledge about how the molecules bind to their targets. The students were also working with a relatively small set of data; neural nets typically perform well only with very large ones."

NNs in general have enjoyed lots of successful practical (commercial) applications in pattern recognition though they were sort of replaced in the "state-of-the-art" by SVMs in many cases until RBMs and DBNs came along. I agree with your caution for skepticism though, only time will tell how good DBNs are.

I think the black box criticism is BS for the most part. In some cases (google's search being a famous example) it might be great to have a human readable and tweakable solution (assuming you have the resources) but for something like recognising handwritten digits from images, not so much.

[1] http://www.nytimes.com/2012/11/24/science/scientists-see-adv...

Regarding the black box criticism, it seems to me that most popular algorithms (SVM, Random forest, ...) become black boxes once you go past the simple 2D example and apply them to real problems. Real-world decisions trees are pretty unreadable and include some rules that really don't make more sense than the weights in a neural network.

> it might be great to have a human readable and tweakable solution (assuming you have the resources) but for something like recognising handwritten digits from images, not so much.

Agree, but with black-box I meant not something that is opaque to my grand-mother but partially opaque to engineers that implement MLP machine learning applications and the tech-lead that takes the decisions. The thing is that even research people (or maybe specially them) tend to positively bias things they do not completely understand (so I think, maybe its just me ;)). That is what I meant with black-box delusion. As you say only time will tell.

Regarding DBNs, again, the case of ASR uses DNNs which is to say big-fat MLPs. The model is handled as a DBN only for pre-training, and layer-wise pre-training does a similar job anyway.

Regarding the "black-box delusion", it's not just you. You see a magician do a trick, and it's amazing. Then he explains how it is done, and the excitement vanishes. Oh, that's all it is, no big deal.

Any sufficiently advanced technology is indistinguishable from magic, and who knows what wonders magic might accomplish? But once you understand the "trick", it's obvious that it can't do much more than what it's doing. Oh, well. The magic is gone.

Reply on news.ycombinator.com