I've been exploring a related concept in a Telegram game. The player can basically create a choose your own adventure-type world by describing a set of interlocking scenes and the segues (buttons/commands) between them.

I'm using Disco Diffusion for the rendering instead of Dall-e 2 but the limitations are similar. It can be frustrating to get consistent results or impose the kind of order on them that is required to make something feel like a real game. Or maybe I'm not thinking hard enough..

If this kinda thing (AI-generated game worlds) interests you, try it here: https://t.me/xiadreamland_bot