I've been thinking about this a while and it's obvious to me:
Put Minecraft (or something similar) under the hood. You just need data structures to encode the world. To enable mutation, location, and persistence.
If the model is given additional parameters such as a "world mesh", then it can easily persist where things are, what color or texture they should be, etc.
That data structure or server can be running independently on CPU-bound processes. Genie or whatever "world model" you have is just your renderer.
It probably won't happen like this due to monopolistic forces, but a nice future might be a future where you could hot swap renderers between providers yet still be playing the same game as your friends - just with different looks and feels. Experiencing the world differently all at the same time. (It'll probably be winner take all, sadly, or several independent vertical silos.)
If I were Tim Sweeny at Epic Games, I'd immediately drop all work on Unreal Engine and start looking into this tech. Because this is going to shore them up on both the gaming and film fronts.