Hacker News Reader

Comment by cba9 | Hacker News Reader

That is... an unusual use of regression, to say the least. When I see something sufficiently off the wall, I always wonder how the authors happened to think of that. It also raises a lot of questions like, how much of physics can be usefully approximated by some random forests? Could you replace most of a physics engine with an appropriate small neural network?

View on HN

Random Forests are part of how the LHC found the higgs, based on my armchair understanding of the slide decks.

Measurement, Monte Carlo and Machine Learning form an interesting triangle. My impression of the LHC pipe is they used monte carlo model sims to train up classifiers that would flag the data relevant for distinguishing alternate models.

The major difference there is that LHC data measures processes which are inherently stochastic (because QM), so using these methods is pretty natural. The OP, however, is applying the same methods to highly complex, but nominally deterministic problems.

I'm curious about this. A lot of us in the computational world have to deal with able but slow iterative methods for things like fluid dynamics and electromagnetic simulations. I'd be interested to know if this works with MHD, and how well it does.

It probably would, but it is harder to get performance guarantees. Seems like a taylor expansion on steroids, but with the taylor expansion you get analytical tractability (to prove correct convergence) along with speed, which you don't get in this case.

Not to hijack the thread, but would you mind posting a link to the LHC slide deck you mentioned?

I watched the announcement live stream. I'm sure the video and slides are up somewhere. Sorry I don't have it at hand.

Check out NeuroAnimator[0] from 1997 (also early work from Hinton). It covers some of this using local-spaced hierarchies of neural networks that predict the deltas for the next time-step.

[0] http://web.cs.ucla.edu/~dt/papers/siggraph97-sketch/

Predictor-correctors is the principal behind a lot of physics engines. I guess they just got a better predictor.

Actually, what they have here is a way of generating predictors from the corrector; which theoretically can be optimized for your domain.

Given that ANNs are universal function approximators, it is natural that one would use them to actually generate a simplified model of simulations. What you see here, taking an already accurate model and generating an approximate implementation, is definitely rare but not unexpected. Long-running simulations can have their run-time drastically compressed simply by developing a neural network capable of being parallelized on GPUs, as in this paper. The ANN run time could be further compressed by generating a single-hidden-layer equivalent. Even though they are known to be exponentially larger than multiple hidden layers, the implementation can be valuable in real-time systems where analysis latency is at a premium, e.g. image recognition in machine control systems.

Neural nets are seen as universal function approximators when you study them in detail. While it isn't the usual "neural net news" you see posted online, this is exactly what ANNs are meant to do.

It occurred to me and I work as a web developer. Just think of things in abstract enough terms and possibilities like this pop out. Having the technical sophistication and time to do the work is another matter.

Reply on news.ycombinator.com