Yeah, "just". Even though we have no idea how to even approach that.
We'll have to cheat somehow.
Sure, this is still effectively magic, but a few years ago I thought we're also anywhere near having the software layer solved - specifically, the Star Trek style computer interaction and holodeck "storyteller" - the thing that would let you create a high-fidelity interactive world with believable characters and a story generated as you play, out of a command like "computer, give me a cafe in Paris, circa 1890". Now we suddenly have all the pieces for that - we can literally just do it, as long as we constrain the medium to just text, with perhaps some generated imagery for extra mood[0]. And I'm not even talking about GPT-4 - I've had a convincing holodeck textual roleplay on AI Dungeon back ~2-3 years ago, back when GPT-3 just came out.
Note: I don't want to hijack this thread and make it into yet another LLM discussion - but I want to point out we have a bunch of parts converging into an entirely new kind of experience. And while we may not crack hard light any time soon, a wearable VR/AR holodeck experience now seems in reach in under a decade. Or perhaps closer to SAO[1] than a holodeck, but still - something that felt way beyond our capabilities just a couple years ago.
--
[0] - Though GPT-4V should be able to play a board game where you send an image of an updated board each turn, shouldn't it?
[1] - Sword Art Online
We do have a bunch of parts, but I don't see any converging happening
But as for sensation for the viewer, no, there’s just what you’d find in a toy shop.