Hacker News Reader

But it will be much more accurate. The neural network will learn to take into account air resistance and wind, for instance. Depending on how much data you give it and the resolution, it might even learn to model air turbulence, spinning and wobbling of thrown objects, etc. It will be a lot more accurate then the simplistic physics model.

And sure, it doesn't output a nice simple equation. But in most real problems, there aren't nice simple equations to output. Physics is sort of a fluke, where the fundamental laws of nature happen to be so simple we can do that. In something like biology, you are never going to find a simple equation that explains a biological system. They are complicated messes of millions of interacting pieces. Even if we knew how everything worked, we don't have the computing resources to accurately simulate it anyway.

And we are actually seeing neural networks be useful in these areas. They are able to predict what chemicals are more likely to be useful drugs. If you restricted yourself to using only simple mathematical models, you would never get these results.

This very article is about the very difficult domain of image processing. Why should we expect the statistical distribution of photographs to be simple and explainable by simple math? Finding an equation that accurately models the shape of a human face is pointless and misguided. Before NNs came along, they were doing crazy stuff like handcoding complicated heuristics that detected dark spots that might be eye sockets. There's nothing elegant about that. Physicists are really blessed to have a domain where they can ignore stuff like air resistance. Where the data really does fit simple equations.

And as the parent comment says, you can interpret neural networks. You can observe the activation behaviors, and discover what features it's learned to detect eye sockets, or chemicals with certain properties. You can see how the output changes as the inputs change, and fit a simpler model to that.

But that's precisely the problem isn't it? It's hard to know when it's picking up additional details or when it's overfitting to noise.

Why is that hard? Expose it to data it hasn't seen yet and see if the results check out.

Expose it to which data exactly? Say it's picking up on some obscure phenomena that is relatively rare, yet very important. How would you, completely unaware of this, be able to collect more relevant data when the majority of the data you collect would not have this element?

But that's a problem with the data, not with the functioning of the net. The whole thing hinges on the premise that you will be able to tell 'sense' from 'nonsense' when you see it, even if you don't know exactly why it labeled 'sense' as 'sense' and vv.

If you are simply amassing data without purpose then what you could do is to try to visualize the layers of the network to see if any surprising features turn up but that will only work if the features are obvious enough to stand out and if that were the case I would suspect we would not have this discussion in the first place.

Computers excel at: remembering stuff forever and speed.

So any kind of improvement that a neural net or any other solution would bring to the table over a human would likely fall in either one of those categories, either the computer is faster at solving the problem, or its ability to remember and apply a vast amount of data to the problem will give it a (slight) edge over what a human could do, or maybe it will be just enough to reach parity.

It will not tell you what is and what is not significant in the input, though, with enough samples you with your slower but much superior intellect might be able to draw new and far-reaching conclusions once you are exposed to the data in sufficient quantity yourself that a pattern suggests itself.

That's in a way a very nice collaboration between man and machine, each doing what they are best at.

A key element in something like this being at play would be the computer consistently being at odds with experts in the field but being right more often than not about those cases. That would be an excellent opportunity to wake up to the possibility that there is something that should be obvious and noticeable but that still got missed.

> Computers excel at: remembering stuff forever and speed.

> So any kind of improvement that a neural net or any other solution would bring to the table over a human would likely fall in either one of those categories, either the computer is faster at solving the problem, or its ability to remember and apply a vast amount of data to the problem will give it a (slight) edge over what a human could do, or maybe it will be just enough to reach parity.

> It will not tell you what is and what is not significant in the input, though, with enough samples you with your slower but much superior intellect might be able to draw new and far-reaching conclusions once you are exposed to the data in sufficient quantity yourself that a pattern suggests itself.

I think that its too early to say how AI will be deployed in practice -- whether it will augment or replace human roles. The ambition is certainly to produce enough "intelligence" to supersede a large fraction of human decision making.

As we seek to make computers more powerful, robust, and versatile ("AI"), it seems like we are pushed towards more organic computational structures. If the trend continues, it would imply that the closer we get to AI, the less their strengths and weaknesses would resemble computers of yesteryear. The interesting possibility is that one might be able to have an interpolation between the strengths of humans and computers.

That may have something to do with the fact that even the biggest neural networks have a "cortex size" one or two orders of magnitude smaller than a cat and "exist" (number of impulses processed * amount of time that would take in an animal) for a few weeks.

That means one should expect them to perform tasks about as well as a mouse brain can do them, between a third and halfway to adulthood.

And yes, that level of cortical processing does not seem to support coming up with symbolic systems and attempting to use them to describe the world around them. Absolutely.

Mice don't do that either.

Am I out of date? I thought that current big deep networks had 100's of millions of parameters, where as mammalian brains were of the order of 10's of billions of neurons with perhaps 1000 times more connections so giving 10's of trillions of parameters (if you imagine a mammalian neuron to work like that).

One parameter maps to one synapse, not to one neuron.

A Neural network will give you a relation which is approximation of that relation not just a numeric output . Also you should remember that while it looks elegant the Newtonian model is also an approximation and fails noticeably in cases where relativistic effects are significant

>But neither the NN nor the researcher will be able to give you a rule like: "F=(G m_1 m_2)/(r^2)" to explain the underlying reasons for the object-trajectory dynamics.

It is not impossible. There is a combination of a special neural network architecture and strong sparsity-inducing regularization that makes it possible to learn equations from dynamics dataset: https://openreview.net/pdf?id=BkgRp0FYe

This is a very interesting paper. Is there an implementation example in any major framework?

It seems to me that you could treat a sufficiently accurate model (be it a deep net or otherwise) as the solution to an unknown, governing physical law, and attempt to identify the law from the model, post-hoc.

I'm not sure if there's been work done in the domain of pattern recognition, but here's an overview where in an example, Navier-Stokes is identified from data:

https://sinews.siam.org/Details-Page/data-driven-discovery-o...

That's the same as my own example of explaining to people that dogs brains can do some pretty complex math. I throw a stick that's turning as it flies and the dog catches it every time, but it can't explain nor communicate the formula that went into that.

But this is really an optimization problem: representing your formula as a bunch of weights that are then used to drive a cascade of multiply-add rules to derive some output is an imprecise but roughly accurate way to model a problem. That there is a much more direct and analytical way of modeling that problem requires intelligence of a different order.

But a program like mathematica gets awfully close to that, so that's not a problem that is a good fit for being solved with a neural network even if it would work.

I'd prefer to use a neural network for problems that have a less well defined solution.

A beautiful example and well-illustrated by the fact that humans could play catch before the development of trigonometry

> But neither the NN nor the researcher will be able to give you a rule like: "F=(G m_1 m_2)/(r^2)" to explain the underlying reasons for the object-trajectory dynamics.

Come on, you can just fit the NN numerical outputs with Excel and come up with tentative explanations for both the underlying elegant rule and the science. The point with NN-based science being you are not left guessing much, instead retrofit the NN numerical output and build a coherent narrative with other accepted results.

Without that equation, you cannot build the rocket that goes to the moon, however.

Feeding the retrofitted "blurred" equation to mathematicians and let them come up with tentative exact models, while being almost there already?

Anything that fits within the error bars is viable if you ask me. Whatever works works.

A lot of investors agree with you. Often they get very rich, especially if they head for the beach before the fit breaks down.

On the other hand...

> cannot give you a generalization

Maybe a second NN could find that for you. You might end up with factors like "speed of an unladen swallow", but perhaps all that is needed is some nudge away from that local maxima to keep searching the space.

>Correct answers don't have to be elegant per se, its nice when they are, but it's not a prerequisite

If a powerful idea is not elegant, we have the option of creating new mathematics in which it _can_ be elegantly expressed.