Hacker News Reader

Comment by manas96 | Hacker News Reader

In my day job I program rigid body behaviour in real time amongst other simulations. I think rigid body contact is hard to learn as it is inherently discontinuous.. something you discover when trying to code a solver.

As such I always use this prompt as a test: "A video of a jenga brick tower falling over as a brick is removed. The physics of each brick must be realistic."

It gave me a video of where bricks suddenly disapper or morph into others[1]. The linked video is after 2-3 iterations of me insisting on realistic physics. If you are just glancing at this, you would believe it is realistic.

That said this is still very impressive and one more step towards .. IDK what. But I am a bit reasurred that at least my job won't be fully replaced with AI :)

[1] https://streamable.com/2em1r3

View on HN

> But I am a bit reasurred that at least my job won't be fully replaced with AI :)

I honestly can't comment with certainty that training from videos alone and whatever tokenization scheme they're using will ever get perfect dynamics.

However it is worth noting that transformers can do a pretty good job at learning dynamics with the right pipeline (not video): https://arxiv.org/pdf/2605.15305 https://arxiv.org/pdf/2605.09196

My point here being that representationally, it might be possible to learn good dynamics without a radically different approach/arch. There are already models that extract 3D tracking points from videos, so they could possibly be leveraged for learning dynamics (which on its own gives precedent for end-to-end approaches also possibly working).

Thanks for the additional reading. I've often thought about LLMs and their ability to represent the physical world with its laws. And always concluded it is not really possible to do so with "just" text tokens and their relations in a latent space. It looks to me there are different approaches being taken to tackle this:

* You could instruct your LLM to interact with a simulator to run experiments and infer behaviour

* You could edit the transformer model and inject spatially relevant data rather than text as is done in above paper

* You could change the architecture to be more condusive for representating a world state. I.e., LeCun's JEPA world model.

* You could further enhance some of the above by using a differentiable physics engine (eg. NVIDIA Newton) to calculate losses directly.

But at the end of the day if a model has any hope to always produce realistic physics, it HAS to learn the laws of nature in some form or other. It looks to me that the next big leap could be achieved by combining the last two approaches.

P.S.: I like discussing such topics. If anyone knows a forum or discord with like-minded people, please let me know :)

> P.S.: I like discussing such topics. If anyone knows a forum or discord with like-minded people, please let me know :)

Unironically twitter (and only use the "Following" tab as opposed to the "For You")

Make an account that only follows university affiliated researchers with less than 1000 followers. In my experience discord servers get suffocated by beginners and crackpots because conversations don't naturally self-organize into their own threads.

Thanks, I'll try using the "Following" tab. I have a lurker account but never really used it because I only ever saw crap in "For You".

I'm not sure why, especially because you're a developer... But damn, the amount of people that expect AI to just one shot stuff is hilarious. Half of the time I make a typo or something, should I be laughed out of the room?

They said the given example took 2-3 iterations. If you think it could be done in 4-5 etc maybe you could share your own result?

I did prompt additional times insisting on realistic physics..

Such videos are essentially dreams: how it feels that the planks should move, not what equations of rigid body physics would compute. And the feeling is realistic (even if overly dramatic in the end). If "stylistic transfer" works for static pictures spread out in space, why won't it work for the character of motion spread out in time?

I wonder what's the training data that makes it generate the final "explosion"...

A little too much Michael Bay

I was thinking eleven.

The physics engine glitching is very realistic :-P

Interestingly, the video on the announcement also starts with some papers and a toy car on a wooden table exploding like those jenga pieces.

Classic 3d simulation artifact with boundary conditions. I remember for an assignment where I had to model liquid with rigid bodies, they would suddenly gain infinite force at the corner and just disappear. It's clear that they must have used a lot of these kinds of synthetic data. But what's impressive to me, every release of these models, I am feeling less and less uncanny valley.

Totally unrelated, but what would you say the feasibility of writing simulation software for simulation of/replicating body movements during/in a martial arts technique would be?

I’ve often thought it would be very handy to have a proper simulator for being able to simulate and identify inefficiencies in one’s technique, but no idea whether it would be feasible to do.

Would be similar to the typical simulations of humanoids. If you need to model the deformations of the human body, or get a proper model of tendons that make up humans, it'll be more difficult, but possible.

Proper simulators for those exist, you essentially need an engine with a compliant contact model. MuJoCo is the goto here, see:

https://mujoco.readthedocs.io/en/stable/modeling.html#muscle... https://mujoco.readthedocs.io/en/stable/computation/fluid.ht...

These explicitly model biological muscles. IIRC it was originally created to model human hands (I could be misremembering though).

Really depends on the fidelity you want.

Edit: I also work in rigid body simulation for robotics.

Indeed, it entirely depends on which axis you want to focus on. A loose trade-off chart would be speed, stability and accuracy. You can only have two of these in a simulator.

Robotics folks probably want speed and accuracy. I'm from the video game industry so I generally look for speed and stability.

Note: This is a loose analogy and recent techniques are already blurring the lines between these axis.

I think modelling accurate articulated body dynamics is feasible but when you add deformation (muscles) it gets much harder.

That[1] video looks very Twin towers. Falls in on itself and then explodes.

thanks for intro to streamable

In my experience (from a couple of years ago), Streamable can be great but it's just worth checking what their current retention policy is like.

We were sharing game clips with each other and after a while realised our old clips were just gone, being deleted after 30 or 90 days or something.

noted!

it was the first link I got after googling free video hosting sites

I guess I haven't tried hosting/sharing anything outside of an unpublished youtube video or GDrive link in a long time.

Some serious clipping

Reply on news.ycombinator.com