Hacker News Reader

Comment by frankmcsherry | Hacker News Reader

There seems to be a lot of angst in the responses about how deep nets et al "are too elegant; are too interpretable!".

It's important to remember that the intended product of science is not tools, it is understanding. Deep nets may produce very elegant and interpretable tools, from a very elegant and interpretable theory, but that is not what the scientists are looking for. They are generally creating and assessing theories in their subject domains, which are then evaluated by the tools they have built.

Deep learning does do a great job of providing a baseline ("your theory must be this accurate to matter"), but it seems to do a much less great job at extracting new understanding.

View on HN

Machine learning has departed from the venture of understanding-based science much before deep learning [1][2]. Thus, singling out deep learning reeks of motivated reasoning or just lack of awareness of the history of machine learning.

Having said that, there still exists the lunatic fringe (it's a compliment) inside deep learning who continue to work on the task of generative modeling in the hope of "understanding the world" rather than "solving a task". Yoshua Bengio and Yann LeCun don't miss an opportunity to impress on the world how important unsupervised learning is.

Physics too has abandoned understanding for predictive power. If you take philosophy of science seriously, you either have to pick "science is whatever the elite society believes in (Kuhn)" or "science gives us tools for prediction (Post-Kuhn)". Claiming that science helps in finding truth or understanding, is going to put you in a very indefensible slippery slope.

"Understanding" is a subjective human concept, and not worth pursuing. Humans evolved to run from tigers, and struggle with harder tasks like understanding quantum mechanics or understanding the brain. The directionality of scientific progress as evidenced by QM is not "understanding", but tooling and predictive power. I hope Deep Learning is going to follow the path of predictive power, rather than regress into the pseudoscientific mess of elegance and understanding.

[1]: http://www2.math.uu.se/~thulin/mm/breiman.pdf

[2]: http://norvig.com/chomsky.html

"Understanding" is a subjective human concept, and not worth pursuing."

Isn't human understanding of how our environment works precisely how you're able to sit in your chair and write comments on HN for others to read?

Sure but I might say that if I were to create a chair by generative methods, would I need to understand how it was created for it to be useful?

If you have ever crafted a chair, table, or anything else that is built by craftsperson, you will soon realise why such a profession exists, because it's very difficult.

You might slap a few pieces of wood together and by brute force, create something like a chair and that might do, but it's very unlikely to be a good, safe, attractive, comfortable long lasting chair. Building quality furniture is complex and difficult work which takes a lot of skill, experience and understanding in order to do.

I understand your point, I never said there is anything wrong with experimenting with NNs, I just don't agree with the OPs sentiment.

No. Tools enabled it, not understanding.

Give one specific example of a widespread modern invention that did not require any understanding to create. You personally can use a tool without understanding how it's made, but you seem to claim the tools are invented without anyone's understanding.

It is fascinating to see in real life, how science-fiction style AI is becoming a reality, where nobody understands how machines produce their answer. Given a few more decades, I'm sure the Asimov short story - The Feeling of Power - will become culturally relevant.

>> Physics too has abandoned understanding for predictive power. If you take philosophy of science seriously, you either have to pick "science is whatever the elite society believes in (Kuhn)" or "science gives us tools for prediction (Post-Kuhn)". Claiming that science helps in finding truth or understanding, is going to put you in a very indefensible slippery slope.

This may be threadomancy, but I kept thinking about your comments and I realise now that there is a huge confusion between the different ways that the word "prediction" is used in the sciences.

What you mean when you say that you can use quantum theory to "make predictions" is that you can plug in some values to the formulae and come up with other values- the position or speed of particles and so on. This is a "prediction" in terms of stochastic determination of the state of the world within a probabilistic framework: you "predict" the values that some random variables will take.

On the other hand, what is more commonly meant by "prediction" in the sciences is the ability to anticipate novel observations, phenomena that have not been observed yet. For a famous example, phycisists and mathematicians predicted the existence of black holes before those were observed by astronomers.

This ability to foresee as-yet unseen phenomena is the true power of scientific theories, the context in which "prediction" is most often used in the sciences, and something that cannot be achieved without a thorough understanding of said theories. You can't take a black-box model of pictures of dog breeds and make a guess about what other kinds of dog breeds might exist, or might be created, than the ones in the original training set. The model may even be capable of doing that- but you, the person training it, are none the wiser. You can't use the model to expand your knowledge about the world. It's a closed loop.

So it's not really "predicting" anything in the way you say it, the way Kuhn meant it or anyone else means it.

Reply on news.ycombinator.com