Hacker News Reader

Comment by jonplackett | Hacker News Reader

What I think could be amazing about ChatGPT integration is (holding my breath…) the ability to iterate and tweak images until they’re right, like you can with text with ChatGPT.

Currently with Midjourney/SD you sometimes get an amazing image, sometimes not. It feels like a casino. SD you can mask and try again but it’s fiddly and time consuming.

But if you could say ‘that image is great but I wanted there to be just one monkey, and can you make the sky green’ and have it take the original image modify it. Then that is a frikkin game changer and everyone else is dust.

This _probably_ isn’t the way it’s going to work. But I hope it is!

View on HN

I am using Automatic1111 and SDXL and this is exactly the workflow people are doing with that.

Generate an image, maybe even a low quality one. Fix the seed and then start iterating on that

do you have a link on how this is done? what does "fix the seed" mean? I was experimenting last night with seed variation (under Extra checkbox next to the seed input box) and i couldn't accurately describe what the sliders did. Sometimes i'd get 8 images that were slight variations, and sometimes i'd get 3 of one style with small variations, and then 5 of a completely different composition with slight variations, in the same batch.

As far as the OP goes, they claim you don't need to prompt engineer anymore, but they just moved prompt engineering to chatgpt, with all of the fun caveats that comes with.

I think by "fixing the seed" they just meant using the "Reuse seed from last generation" button. By default, this will mean that all future images will get generated with the same seed, so they'll look identical. The "variation seed" thing mixes slight variations into this fixed seed, meaning that you're likely to get things that look similar, but not identical to the original. The "variation strength" slider controls how much impact from the variation seed is mixed into the original, with 0 being "everything from the initial seed", and 1 being "everything from the variation seed". I'm pretty sure that leaving the width/height sliders at their default positions is fine.

Also, just for future reference - lots of UI elements in A1111 have very descriptive tooltips, they help a lot when I can't quite remember the full effects of each setting.

Yes, that describes it pretty well.

By default the random seed is different for each generation, so even if the prompt and every setting is the same you'll get vastly different images ( which is the expected outcome ).

When you set the seed to a fixed number you get the same image every time you generate if all other settings and the prompt are exactly the same.

So I start off with a pretty high CFG Scale to give it enough room to imagine things ( Refiner and Resize turned off )

Once I have a good base image, I'll send it to img2img and iterate some more one it.

Thanks for info. I don't use img2img except when i want to "AI" a picture, but this makes sense, as one of the steps. I usually just "run off"* dozens of images and pick the best one - but i don't do this for a living or anything, just to be able to "find" a picture of something in less than a minute.

* like a mimeograph!

Not quite. In A1111, the process is "pick a seed and iterate the prompt". You're still regenerating the image from noise, and if SD decides to hallucinate a feature that wasn't in the prompt, it's often very hard to remove it.

Since gpt4 natively understands images, there's the potential for it to look at the image, and understand what about it you want to change

Exactly - iterative "chat-driven" improvement of images will be a paradigm shift

That looks exactly like what was in the video (though the video seems to be down?)

Didn’t see a video! OK I am pretty excited now.

So long as it actually can create an image virtually the same but changed only how I want it. That would just blow everything else away.

I mean I guess it’s. Or that ridiculous. Generative fill in photoshop is kind of this, but the ability to understand from a text prompt what I want to ‘mask’ - if that’s even how it would work - would be very clever

You can _kind_ of do this now, but it's excessively manual. I agreed that it would be awesome to have the AI figure out what you mean to iterate on a given image.

I recently had a somewhat frustrating experience of having a generated foreground thing and a generated background that were both independently awesome - and trying to wheedle Stable Diffusion into combining them the way I wanted was possible, but took a lot of manual labor, and the results were "just ok".

Reply on news.ycombinator.com