I feel bad for being dismissive of their work, but... a lot of it is just low quality and predictable filler.
We'd likely need consoles to pave the way with AI cores on their SOCs, then we might get new pcie AI cards and that's probably the first time the feature could be leveraged properly.
Doing everything in the cloud would be prohibitively expensive and likely too slow, as 15 second would be just too long for a player to stand around waiting for an answer.
Though the best way to do interesting worldbuilding IMO is look up a few cultures less represented in fiction (or at least parts of them that are less represented) and study their history to get ideas. I've been reading up on ancient Greece lately and getting a ton of ideas.
I think I enjoyed the back and forth, the ability of me giving the rough sketches of the story and scene and then the bot filling in and adding some details. Then I could use those details to further the story.
It's still not there, but mostly because ChatGPT is too terse and biased towards everything being OK. Nonetheless, it is a lot of fun to have a story-telling copilot.