Yeah reading this I was thinking, we've got Qwen-Image-Edit which is an image model with an LLM backbone that takes well to finetuning.
I'd be surprised if you can't get a 80%/20% result in a weekend, and even that probably saves you some time if you're just willing to pick best-of-n results