Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past a tech demo (and for some reason all the "ai game" people do this approach).
Dunno, this seems like an avenue definitely worth exploring.
Plenty of game applications today already have a render path of input -> pass through AI model -> final image. That's what the AI-based scaling and frame interpolation features like DLSS and FSR are.
In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps.
Experiments like the OP's are about moving that boundary and "prompting" with a lower-fidelity input and having the model do more.
Depending on how well you can tune and steer the model, and where you place the boundary line, this might well be a compelling and efficient compute path for some applications, especially as HW acceleration for model workloads improves.
No doubt we will see games do variations of this theme, just like games have thoroughly explored other technology to generate assets from lower-fidelity seeds, e.g. classical proc gen. This is all super in the wheelhouse of game development.
Some kind of AI-first demoscene would be a pretty cool thing too. What's a trained model if not another fancy compressor?
> Plenty of game applications today already have a render path of input -> pass through AI model -> final image.
Where "plenty" equals zero?
> In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps.
That is, in these cases you already have high-fidelity input in the form of an actual game, and "AI" contributions to the output are dubious at best.
Do you really believe it's DLSS that's doing all the heavy lifting in a game like Expedition 33 or Cyberpunk 2077?
And no if you heavily visually modify something with AI models to the extent it significantly alters the appearance it simply has no way of being consistent unless you include the previously generated thing somehow which then has the huge problem of how do you maintain that over an 80 hour game? How do you inform the AI what visual elements (say text, interactive elements) are significant and can be modified and which can't? (You can't)
Actually using AI to generate assets, having a person go in to make sure they look good together and make sure they match then just saving them as textures so they function like normal game assets makes 10000x more sense then generating a user image then trying to extract "hey what did the wrapper of this candy bar look like" from one ai generated image and figuring out how to make sure that is consistent across that type of candy bar in the world and maintains that consistency throughout the entire game, instead of just you know, generating a texture of a candybar?
That being said, it took us a few hundred years all in all just to work out paint, so if people keep working with this tech eventually a game designer could, in theory, lay out the skeleton of a game and tell an AI to do the rest, give it pointers and occasional original art to work into the system, and ship a completed playable game in days.
Whether it will be worth playing those games is an entirely different enchilada to microwave.
Like with that tech what kind of games would say random solo developers plugging at it and refining it be able to make in 4 years, that is the extremely compelling stuff. One person being able to make some auteur AAA quality game on their own, even if it takes a long time that might actually be good. If there are AI games those are the ones I'd want to play.
"But, think of the indie game designer!" is getting to be quite the take.
We have a machine that produces slop and the selling point is how fast it produces it? And how more people should be using it to spend less time on creative aspects? Would the world be a better place if GRRM "finished" his most well-known work sooner rather than never?
Something about the phrase "tell an AI to do the rest, give it pointers" reminds me of "The Sorcerer's Apprentice" from Fantasia. Not in the surface level dire-warning about laziness, automation, and losing control that story is telling but in that Mickey didn't spend any time thinking about what he was doing and the Wizard's disappointment at the end.
Here’s a video from a rando on Reddit conveniently posted today after playing around for an afternoon: https://www.reddit.com/r/aivideo/comments/1n1piz4/enjoy_nano...
The Nvidia team carefully selected a scene to be feasible for their tech. The rando happened to select a similar-ish scene that is largely well suited for game tech.
Which scene looks more realistic? The rando’s by far. And, adding stylization is some AIs are very well established as being excellent at.
Yes, there are still lots of issues to overcome. But, I find the negativity in so many comments in here to be highly imbalanced.
Also, how are these 2 comparable at all? Obviously the video looks more "realistic", the first one is obviously a game demo of some kind and is stylized, whereas the latter looks like a terrible travel agency ad.
Don't focus on the emotional context of the scenes. Look at the physical content. They are both contain stonework, plants and a human. They are both large, detailed scenes lit by strong sunlight and a bright sky. As far as rendering technology requirements go, they are very similar.
> scratches my brain in a very unpleasant way, it's hard to describe the sensation. The closest thing is "disgust"
There is strong discontinuous motion effect because of how the video tech is based on a sequences of "first frame, last frame" inputs spliced together. There are a couple seconds where her behavior gets uncanny valley --particularly her hands on the door. Wouldn't be a concern in an unrealistic video game. But, would be in a this-realistic game regardless of the tech. And, there is a very slight warble in the fine details. But, you have to really look for those to distinguish them from MPEG artifacts.
I expect what a lot of people here are feeling when they watch the video is disgust based 90% or more just on the fact that the video was AI generated :/
It makes no sense when people say AI can't do this or that. It will do it next week.
So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.
Self-driving cars are "here"... until someone adds another requirement.
Yes, Waymo can today drive around extremely dense car-friendly cities that are scanned and mapped in great detail weekly... They also still have to have remote human intervention all the time, and are freaked out by traffic cones being placed on the hood. I grew up in Indonesia and that's where I learned to drive, and trust me, if Waymo is ever able to navigate 100 meters on any road in Jakarta I'll happily concede and consider self-driving to be a solved problem.
In fact, I am not arguing that self-driving cars are perfect or global. I am pointing out how people keep changing the definition of "solved" which makes it look like the finish line keeps moving.
We do have what parent said, it is a reality. It is also the reality that it is not perfect, but somehow the latter is made to minimize or completely dismiss the former.
Those are the goalposts and they've been in the same spot for quite a while now.
I just do not think that your response is a valid response to a fact ("Waymos are driving themselves around several cities right now").
The failure mode for getting a self-driving car right is grave. The failure mode for rendering game graphics imperfectly is to require a bit of suspension of disbelief (it's not a linear spectrum given the famous uncanney valley, etc., I'm aware). Games already have plenty of abstract graphics, invisible walls, and other cludges that require buy-in from users. It's a lot easier to scale that wall.
The statement was one of capability. There are some things that the tech is flatly not capable of, and that it will take time to develop the capability of. Even if there were no safety concerns at all and we lived in a cotton candy bubble world, self driving cars still have hard failure modes. The tech is not capable, and will not develop the capability next week, either.
The point being made is that the tech is moving fast, at least according to the marketing, but a revolution is not happening ever week. "This is the worst it'll ever be" is an increasingly tired refrain when things seem to be stagnating more than ever. The mentioned behavior will take longer a good amount of time, it's silly to wait around for it when it is not unlikely it may never come.
The problem with these statements is language has so much context implicit in it. "driving around on their own" to me means with zero active oversight. "driving around" to me means not just in a small set of city streets, but as a replacement for human driving (eg anywhere a vehicle can physically fit). Obviously to you it means other things, but it's what makes these conversations and statements of fact challenging.
Tesla's have >1 but they are not really self-driving, but more "100% human supervised self-driving."
There have been musings for a while now that 3D rendering is going to switch from “lay down the scene’s albedo & specular parameters then do the lighting pass” to “lay down the scene’s latent parameters and then do the diffusion pass”.
Recently, the advances in “real time AI world models” have been coming ridiculously fast.
Put these together and it’s no stretch at all imagining a game built by having artists go nuts doing whatever they want with whatever Maya can handle as long as they also make proxy geometry of trivial complexity that can be conceptually associated with the final renders. Train the AI on the association. Render the proxy geometry the old fashioned way. AI that up in real time to the associated Maya-final-render approximation.
It’s not going to happen this week. But, in 5 years? Somebody’s gonna pull it off.
in a tech demo, sure. In a real product? I doubt it. We have neat tech demos from 20 years ago that amounted to not much, because implementing it in a real business product was unviable, unweildy, or didn't meet standards.
There's a lot to get away with with demos. Not so much in a product you want to sell. Much less so for a deluxe product with a bunch of competition like games.
You're gonna have to break the light barrier before thinking about how to do real time Ai rendering at this rate.
Genie 3 is incredible (relative to previous models) in this regard. Not a solve, but it is doing what it is not supposed to be able to do.[1]
[1]https://deepmind.google/discover/blog/genie-3-a-new-frontier...
I think the point of this is just because it's cool. So, as you said, it only serves as a tech demo, but why not? Many things have no point. It's unreasonable, but it's cool.