Comment by frumiousirc | Hacker News Reader

Comment by frumiousirc | Hacker News Reader

A fundamental problem with this entire class of machine learning is that it is based on a model / simulation of reality. "RocketPy, a high-fidelity trajectory simulation library for high-power rocketry" in this case.

Nothing against this sim in particular but all such simulations that attempt to model any non-trivial system are imperfect. Nature is just too complex to model precisely and accurately. The LLM (or other DL network architecture) will only learn information that is presented to it. When trained on simulation the network can not help but infer incorrectly about messy reality.

For example, if RocketPy lacks any model of cross breezes, the network would never learn to design to counter them. Or, if it does model variable winds but does so with the wrong mean, or variance, or skew (of intensity, period, etc) the network can not properly learn and the design will not be optimal. The design will fail when it faces reality that differs from model.

Replace "rocket" with any other thing and you have AI/ML applied to science and engineering - fundamentally flawed, at least at some level of precision/accuracy.

At the least, real learning on reality is required. Once we can back-propagate through nature, then perhaps DL networks can begin to be actually trustworthy for science and engineering.

I don't think it's a "problem" as much as it is a "tradeoff". You basically have two approaches to take here: 1) try to simulate as best as you can, iteratively improve the simulation space after trying it out in real-life, and go back and forth or 2) skip the simulation step and do the same process but only in real-life, not having any simulation step at all and only rely on real scenarios, but few of them.

Considering how fast you can go with simulations vs real launches, I'm not surprised the took the first approach.

> A fundamental problem with this entire class of machine learning is that it is based on a model / simulation of reality… all such simulations that attempt to model any non-trivial system are imperfect

Depends on what your goal is. If you are trying to solve the narrow problem of rocketry or whatever, sure. But maybe not if your goal is making models smarter.

The broader context is that we need new oracles beyond math and programming in order to exercise CoT models on longer planning-horizon tasks.

In this case, if working with a toy world model lets you learn generalizable strategies (I bet it does, as video games do too) then this sort of eval can be a useful addition.

> all such simulations that attempt to model any non-trivial system are imperfect.

I believe the future of such simulation is to start from the lowest level - ie. schrodinger's equation, and get the simulator to derive all higher level stuff.

Obviously the higher level models are imperfect, but then it's the AI's job to decide if a pile of soil needs to be simulated as a load of grains of sand, or as crystals of quartz, or as atoms of silicon, or as quarks...

The AI can always check its answer by redoing a lower level simulation of a tiny part of the result, and check it is consistent with a higher level/cheaper simulation.

> I believe the future of such simulation is to start from the lowest level - ie. schrodinger's equation, and get the simulator to derive all higher level stuff.

I do hate to burst your bubble here but I've been doing real-time simulation (in the form of games, 2D, 3D, VR) for enough decades to know this is only a pipe-dream.

Maybe at the point when we have a Dyson sphere and have all universally agreed upon the principles that cause an airfoil to generate lift this would be possible, otherwise it's orders of magnitude beyond all of the terrestrial compute that we have now.

To quote Han Solo, the way we do effective and convincing science and simulation now is ... "a lot of simple tricks and nonsense."

I don't think it's a pipe dream from an 'amount of compute' perspective.

Any competent person can simulate 100 atoms in a crystal of some material, and say "whoa, it seems the bulk of this material behaves like a spring with f=kx, lets replace the individual atom simulation with a bulk simulation which is computationally far cheaper", and then we can simulate trillions of atoms.

I don't see why AI couldn't do the same.

One trillion atoms of the heaviest element is less than a nanogram. I get your point it's just that we can't even simulate all the blades of grass on a one acre lawn with every shortcut we have.

Really I think it would be cool to explore -- I've been working on a procedural game engine (conceptually at least) for a long time and want to incorporate even "basic" things like chemistry. I think it's still decades away for that, not even considering quantum phenomena.

Accurate simulation is also an AI problem, but that should be a separate paper

Reply on news.ycombinator.com