Yeah, you could certainly tell it to skip the Gemini image gen step and replace with generated SVG art, or an existing image, or just stick a longer description there, whatever. It's flexible.
I thought about the deterministic generation aspect, but there's no "random seed" equivalent for frontier LLMs to my knowledge that would guarantee deterministic output (even temperature=0 is still nondeterministic).