Comment by echelon | Hacker News Reader

Comment by echelon | Hacker News Reader

Parent Full thread

Whoa, whoa, whoa. That's just one angle. Please don't bin that as the only use case for "world models"!

First of all, there are a variety of different types of world models. Simulation, video, static asset, etc. It's a loaded term, just as the use cases are widespread.

There are world models you can play in your browser inferred entirely by your CPU:

https://madebyoll.in/posts/game_emulation_via_dnn/ (my favorite, from 2022!)

https://madebyoll.in/posts/world_emulation_via_dnn/ (updated, in 3D)

There are static asset generating world models, like WorldLabs' Marble. These are useful for video games, previz, and filmmaking.

https://marble.worldlabs.ai/

I wrote open source software to leverage marble for filmmaking (I'm a filmmaker, and this tech is extremely useful for scene consistency):

https://www.youtube.com/watch?v=wJCJYdGdpHg

https://github.com/storytold/artcraft

There are playable video-oriented models, many of which are open source and will run on your 3080 and above:

https://diamond-wm.github.io/

https://github.com/Robbyant/lingbot-world

There are things termed "world models" that really shouldn't be:

https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0

There are robotics training oriented world models:

https://github.com/leggedrobotics/robotic_world_model

Genie is not strictly robotics-oriented.

The entertainment industry, as big as it is, just doesn't have as much profit potential as robots and AI agents that can replace human labor. Just look at how Nvidia has pivoted from gaming and rendering to AI.

The other examples you've given are neat, but for players like Google they are mostly an afterthought.

Robotics: $88B TAM

Gaming: $350B TAM

All media and entertainment: $3T TAM

Manufacturing: $5T TAM

Roughly the same story.

This tech is going to revolutionize "films" and gaming. The entire entertainment industry is going to transform around it.

When people aren't buying physical things, they're distracting themselves with media. Humans spend more time and money on that than anything else. Machines or otherwise.

AI impact on manufacturing will be huge. AI impact on media and entertainment will be huge. And these world models can be developed in a way that you develop exposure and competency for both domains.

edit: You can argue that manufacturing will boom when we have robotics that generalize. But you can also argue that entertainment will boom when we have holodecks people can step into.

The current robotics industry is $88B. You have to take into account the potential future industry of general purpose robots that replace a big chunk of blue-collar work.

Robots is also just one example. A hypothetically powerful AI agent (which might also use a world model) that controls a mouse and keyboard could replace a big chunk of white-collar work too.

Those are worth 10's of trillions of dollars. You can argue about whether they are actually possible, but the people backing this tech think they are.

Not so sure around gaming. While it opens some interesting "generate quest on demand" and "quick demo" cases, an infinite world generator wouldn't really vibe with people.

They would try it once, think its cool and stop there. You would probably have a niche group of "world surfers" that would keep playing with it.

Most people do not have an idea on what they would want to play and how it would look like - they want a curated experience. As games adapted to the mass market, they became more and more curated experiences with lots of hand-holding the player.

Yeah, a holodeck would be popular, but that's a whole different technology ballpark and akin to talking about flying cars in this context.

This will have a giant impact on robotics and general models tho, as now they can simulate action/reaction inside a world in parallel, choosing the best course, by just having a picture of the world and probably a generated image of the end result or "validators" to check if task is accomplished.

And while robotics is $88B TAM nowadays, expect it to hit $888B in the next 5-10 years, with world simulators like this being one of the reasons.

From the team side, gotta be cool to build this, feels like one of those things all devs dream about.

Reply on news.ycombinator.com