This is 300M parameters model (1/1300th of the big llama-3) trained with 5M frames with 12 days of a GTX4090.
This is what a big tech company was doing in 2015.
The same stuff at industrial scale à la large LLMs would be absolutely mind blowing.
This is what a big tech company was doing in 2015.
The same stuff at industrial scale à la large LLMs would be absolutely mind blowing.
5 million frames of video data with corresponding accelerometer data, and you get this for genuine photorealism.
And I thought I was fairly explicit about video data, but just in case that's ambiguous: the stuff you record with your phone camera set to video mode, synchronised with the accelerometer data instead of player keyboard inputs.
As for output, with the model as it currently stands, I'd expect a 24h training video at 60fps to be "photorealisic and with similar weird hallucinations". Which is still interesting, even without combining this with a control net like Stable Diffusion can do.
You see, there are these things called "proof of concept"s that are meant to not be a product, but instead show off capabilities.
Counterstrike is an example, meant to show off complex capabilities. It is not meant to show how the useful thing of these models is to literally recreate counterstrike.
If that capability is possible, then it could be possible to take 100 examples of seperate world models that exist, and then combine those world models together in interesting ways.
Combining together world models is an obvious next step (IE, not showed off in this proof of concept. But it is a logical/plausible future capability).
Having multiple world models combined together in new and interesting ways, is almost like creating an entirely new world model, even though thats not exactly the same.
I don't have a problem with you using this with a Camera and the real world or your creations ; but I do have a problem when people are able to use someone's work and use these blend and call them original.
It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
No it's f* not.
Thats undecided by the courts. Everyone is training using other people's data right now though, and very few companies are even being sued (let alone have finished the multi-year process to actually be punished for it).
> Or does that mean I should close off all of my creations
When you put something out there, you should expect that other people are going to use those creations. Almost certainly in many ways that you don't approve over.
Don't release something publicly if you don't want other people to use it.
> but I do have a problem
Your anger isn't particularly useful. What are you going to do about it? This particular proof of concept model was made with 1 single GPU running for 2 weeks. You can't stop that.
> It's just as I would have taken a 3d model static mesh, apply some blend on it and call it my own.
Something that I am sure many people are doing all the time, already. Transforming and using other people's content is as old as the internet. AI does little to change that.
We all thought Sports-Game $CURRENT_YEAR or CoD/MW MicroTransaction-Garbage was bottom of the barrel, a whole new world of fresh hell awaits us!
You have it all wrong. It's not going to be AAA studios doing this. Instead, it will be randos who have a cool/fun concept that they want to try out with a couple friends.
Sure, most will be bad. But there might be some gems in there that go viral.
Making it easier for regular people to experiment with games is a good thing.
Or even better, what if this allows people to throw something together to see if a game mechanic is fun, and once it's tested out, then they can make the game for "real".
Allowing quicker experimentation times is also a good thing.
The rate of progress has been mind-blowing indeed.
We sure live in interesting times!