Hacker News Reader

Comment by accrual | Hacker News Reader

This seems unprecedented. Imagine if you have this but you can update the scene programmatically. Ask your AI to change the location or actors. Now you have a very convincing artificial scene with anything you can imagine in it.

View on HN

With all the various more specialized demos I've seen, that day doesn't seem too far away.

I imagine that most people will have the capacity in their home machines to render stuff like this in video games in 10ish years.

10? I think we'll both be surprised by where we're at in 3.

That's October, 2026.

Re-reading the paper, I totally missed the hardware that they did this on, which was consumer-level GPUs, so I think you may be right - 3 years is probably a good time frame for seeing this kind of tech in commercial games.

My reasoning for initially saying 10ish years:

GPU architecture release cadence is frequent, but not THAT frequent. NVidia released the Ampere (RTX 30X0) in 2020, 3 years ago. Ada Lovelace (RTX 40X0) was released a year ago, in Oct 2022. It's _possible_ to use 4090s to do medium neural-network things right now, but to be on the leading edge, you need multiple datacenter-level cards, each of which is > $5k. Even though it's possible to do crazy things with the most recent generation of GPUs, there don't yet exist games that really take advantage of it. The closest that I'm aware of is the Cyberpunk-level games that make native use of raytracing capabilities.

I think it'll probably be a year or two before we see games come out that really require the level of 40-series cards ( or 7900-series if you're of the AMD persuasion), which is a lag-time of ~3 years after the cards were released. I think the software development time and market saturation are the driving factors in the gap here.

I was under the mistaken impression that the video output was produced on the datacenter cards. They got reasonable performance on a 30-series NVidia card. In 3 years, it's totally reasonable to expect that AAA game players will have that level of GPU performance in their gaming machines, so yeah, I think you're right.

Developers target where the market is. Until 2027 or so, that will be the PS5, which is somewhere around RTX 3060 or 2080 I think.

I imagine this would be helpful when making movies if you could basically play around with the scenes without having to refilm it several times to get the best one.

When it comes to perspective and the like, they already do this; multiple camera angles, CGI, and the odd reshoot. Like having Henry Cavill come back for a reshoot, then CGIing out the mustache he had for his next role.

Between this and LLMs, we're half-way to building a holodeck. What's missing at this point is just hard light - i.e. the being able to feel the physical substance of simulated objects, and being able to experience it all without a wearable/personal device.

> What's missing at this point is just hard light

Yeah, "just". Even though we have no idea how to even approach that.

> Even though we have no idea how to even approach that.

We'll have to cheat somehow.

Sure, this is still effectively magic, but a few years ago I thought we're also anywhere near having the software layer solved - specifically, the Star Trek style computer interaction and holodeck "storyteller" - the thing that would let you create a high-fidelity interactive world with believable characters and a story generated as you play, out of a command like "computer, give me a cafe in Paris, circa 1890". Now we suddenly have all the pieces for that - we can literally just do it, as long as we constrain the medium to just text, with perhaps some generated imagery for extra mood[0]. And I'm not even talking about GPT-4 - I've had a convincing holodeck textual roleplay on AI Dungeon back ~2-3 years ago, back when GPT-3 just came out.

Note: I don't want to hijack this thread and make it into yet another LLM discussion - but I want to point out we have a bunch of parts converging into an entirely new kind of experience. And while we may not crack hard light any time soon, a wearable VR/AR holodeck experience now seems in reach in under a decade. Or perhaps closer to SAO[1] than a holodeck, but still - something that felt way beyond our capabilities just a couple years ago.

[0] - Though GPT-4V should be able to play a board game where you send an image of an updated board each turn, shouldn't it?

[1] - Sword Art Online

> I want to point out we have a bunch of parts converging into an entirely new kind of experience

We do have a bunch of parts, but I don't see any converging happening

I believe the way to go is hijacking neural connections, something between Neuralink and STEM from Upgrade.

Just plug into the Matrix ;)

With all the money generated by the porn industry I'm surprised there isn't more spent on research and development of applicable technology.

There is sort of a hardware arm to that industry. They’ve made it so you can send money to a model who is on stream and your “donation” will trigger a mechanical actuator that …does things…

But as for sensation for the viewer, no, there’s just what you’d find in a toy shop.

So... you can build something out of Lego?

I suppose you could build up what would essentially be a 4D sprite sheet or animation set of a character and use that to support natural looking arbitrary movement. I'm not sure that isn't just a mo-cap character with extra steps, though.

Even the most skilled animators with years of budget still can’t escape the uncanny valley which is why CG animation has converged on a style of blob-humans as the current standard.

I have very little hope of AI driven animation looking ok in the next many decades. Don’t underestimate how hardwired your senses are at finding artifacts in movement. Static images are much easier to “fake”.

Reply on news.ycombinator.com