I've also found that some words and phrases will overpower the rest of the prompt, so it can be hard to get to specific areas of the latent space.
I think they're really good for mood board kind of work and exploring ideas, but you'll probably still need an artist to create something that is specifically what you want.
One shot random prompt stuff is better on Midjourney and I'm still subscribed to it, but I would say I do 99% of my playing around inside StableDiffusion these days.
Using a simple drawing of single colour shapes as the original image and a good prompt, one can get quite far in terms of composition.
The desert dome city still blows my mind every time I think about it.
The UI/UX of a proper workflow is still very janky (even in user-friendly UIs like auto1111), but you can get very specific, complicated scenes with a mixture of ControlNet + inpainting.
> I've also found that some words and phrases will overpower the rest of the prompt, so it can be hard to get to specific areas of the latent space.
I've also had this problem (particularly with multiple colored objects, like "blue eyes, brown hair"). Apparently Cutoff (https://github.com/hnmr293/sd-webui-cutoff) is very good at addressing this "leakage", but I haven't implemented it yet.
Right now, things are really good for mood board kind of work, but there's a lot of tech already available for enabling a lot of back-and-forth "work" on images to get them into exactly what you're imagining. There aren't a lot of good UIs for them all yet though; as soon as they get a little more user-friendly, I'd expect another boom in generative AI for artists (this time, with artists being the primary benefactor).
SD with Multidiffusion or another multiple prompts with (potentially overlapping or nested) regions tool, plus some inpainting, can do a lot here.
So can SD with ControlNet, to go beyond text prompting.