Hacker News Reader

Comment by wasyl | Hacker News Reader

Is there some comprehensive source about how to make the most of Stable Diffusion? I find the examples on websites much better than what I've been able to generate — they more closely convey the prompt and have less artifacts/clearly messed up parts

View on HN

When people call themselves "prompt engineers" it's only half in jest. Half of generating something good is guiding the program into generating something good. That means knowing the right keywords to get specific styles or effects, a little bit of luck, and sometimes generating a prompt several dozen times and then creating variations from a seed once you find a specific seed that generated something close to what you liked. It's an iterative process and many of the fantastic images you see weren't "first generations" but likely the 20th or so generation after tons of trial and error working around a specific prompt/idea.

I'd recommend keeping a prompt list and finding what does/doesn't work for what you're after. Try shuffling the order of your prompt - the order of the tokens does matter! Repeat a token twice, thrice, hell make a prompt with nothing but the same token repeated 8 times. Play around with it! If you find an image that's very close to what you want - start generating variations of it. Make 20 different variations. Make variations of the variations you like best.

Also the seed is very important! If you find a seed that generated a style you really liked take note of it. That seed will likely generate more things in a similar style for similar enough prompts.

It's a semi-creative process and definitely takes some time investment if you want great results. Sometimes you strike gold and get lucky on your first generation - but that's rare.

If someone turns artist names and the quirky-but useful bits of prompting like 'Unreal Engine' as an image sharpener into a Mac app with Instagram style filters they'll make some money...

each engine is a little different as well. it's like learning to perform with a partner - like another dancer, musician, etc. you have to find the sweet spot where what you want and the tool *can do* line up.

I search Lexica.art for the style I want, copy the prompt associated with the work and edit it my needs.

The reddit forum for StableDiffusion has a tag for prompts where you can get a large number of detailed examples to use:

https://www.reddit.com/r/StableDiffusion/?f=flair_name%3A%22...

Also, this post refers to a large number of relevant tools to use as well:

https://www.reddit.com/r/StableDiffusion/comments/xcrm4d/use...

Agreed and wondering myself, DALL-E seemed to do a better job of great looking images with brief prompts, but Stable Diffusion seems to need more specific prompts. SD is free though so would love to use it more.

CLIP-guided Stable Diffusion, or Dalle+SD, are both doable with current open source and will have much smarter prompting at the cost of even more memory use.

I've found the stuff in the DALL·E 2 Prompt Book also works well for Stable Diffusion

https://dallery.gallery/the-dalle-2-prompt-book/

If one prompt doesn't work, try writing it in another way. Sometimes it helps to write things in multiple ways in the same prompt.

Reply on news.ycombinator.com