Show HN: I made 7k images with DALL-E 2 to create a reference/inspiration table | Hacker News Reader

Show HN: I made 7k images with DALL-E 2 to create a reference/inspiration table

generrated.com

96 pointsdavidbarker36 comments

This is not (yet) worthy of a "Show HN", so I'll drop it in here: If you are using Discord and want a low-barrier way to play around with Stable Diffusion, you can use my Bot to do so.

Use https://discord.com/api/oauth2/authorize?client_id=101337304... to invite the bot to your Discord server, or join my demo server at https://discord.gg/nsfeutx35z.

Disclaimer: while you can use the bot for free 5 times, it is not a completely free service; while I don't plan to turn this into a money-making endeavour, I cannot afford to keep the rather costy GPU instance running without outside funding.

Therefore, the bot comes with a credits system - you can buy credits on https://pay-a-robot.com, for $0,05 per credit. 1 credit = 1 text-to-image run with 3 result images.

If you want to host the bot yourself, the source code with a step-by-step setup guide is available at https://github.com/manuelkiessling/stable-diffusion-discord-....

End of the day, unless it's opened up Dall-E 2 will be seen as an evolutionary dead end of this tech and a misstep.

It's gone from potentially one of the most innovative companies on the horizon to a dead product now I can spin up equivalent tech on my own machine, hook into my workflow and tools in an afternoon all because Stable Diffusion released their model into the wild.

Yeah, until Stable Diffusion became available, I felt that Dall-E 2 stance on not opening it up was sorta reasonable. Mostly because "groundbreaking tech producing all these impressive results that cost a ton to build, and I bet stable diffusion announcements were all just riding the hype, and it will disappoint at the end."

I have never eaten my words as fast as I did when stable diffusion had finally released. Such a gamechanger while running locally, it isn't even funny. All these parameters and samplers one could play with, use a one-click simple gui or cli or a web front-end or even hook it up to any existing code flow that you have going. And it all just works well, with every week bringing new advancements (like img2img).

I haven't heard anyone talk about Dall-E 2 in over a month. Like, at all. All while stable diffusion overtook pretty much every single social circle related to image generation that I was in.

Major props to the stable diffusion team. They had a very high bar to reach, and not only they managed to do it extremely fast, they blew way past it. Leading by example, in the face of all the "no no no, we gotta keep the model and everything closed, and you shouldnt be able to run it locally, oh and also we hardcoded input filters because safety and we know better than you do" bs arguments was extremely satisfying to watch.

My stable diffusion output looks awful. I've been trying to recreate the xkcd about Joe Biden eating sandwiches, so I try something like "Joe Biden eating a sandwich in the oval office, 4k render, photograph" and I get nightmare fuel with pieces of bread attached to his head, faces that dissolve into random geometric shapes, toppings that melt into hands while a sandwich sits on a plate in front of him, etc.

I had high hopes based on posts from the SD subreddit, and I figured Biden would be well represented in the training data. Am I missing something?

You might try using image to image with the comic and previously mentioned prompt as inputs.

SD isn't great at generating images for detailed, weird prompts (at least not compared to DALL-E2). If you're not great at prompt writing or just having bad luck, you can use img2img with a rough sketch of what you want.

Here, a less specific prompt:

"Portrait of Joe Biden in the oval office, 4k render"

First attempt: https://pasteboard.co/IYo5m6KeaqF4.png

OK, I'll grant you that all of the parts of a face are there and reasonably correct (except for the bottomless pits of darkness in his nose and mouth)

Second attempt, I end up with these weird artifacts in his head half of the time (3/5 of my generated images)

* https://pasteboard.co/xDFv9KD7Or4U.png

* https://pasteboard.co/j7PqHPgorZ9G.png

Am I holding it wrong somehow?

What is your guidance scale number, the number of iterations, and the chosen sampler? Those would be very relevant to know. Pretty much the most relevant thing aside from the prompt itself.

Setting guidance scale number higher typically results in imagery getting trippier and more surreal with more artifacts. So i feel like that's the main culprit for the artifacts.

I am pretty curious to see how far we can get with this prompt. So I will try playing with it later today and post the results and what I found in a reply to this comment.

I'm using the default settings on the webui, here are the parameters:

``` Portrait of Joe Biden in the oval office, 4k render seed:1331361607 width:512 height:512 steps:50 cfg_scale:7.5 sampler:k_lms ```

Thanks for providing the seed, because that would let me show you how exactly the parameters can affect your specific image without generating a "random" different one every time.

Check out the exact same seed and prompt and cfg_scale, but with the steps aka iteration number at 100 (50 in general feels way too low, even for the samplers that are kinda good with low iteration numbers).

https://pasteboard.co/I6yXg5mZip6D.png

Obvious glitchiness in the face. Below is the same one, but with a k_euler_a sampler (I don't use k_lms, mostly k_euler_a or k_dpm_2_a) + 100 iterations.

https://pasteboard.co/xaTJiN6eVhm2.png

Less glitchiness, but Joe looks more caricature-like, than real. And also, not super quite like Joe. Let's try the same, but at 150 iterations and set the CFG at 10.

https://pasteboard.co/a9OigPXS9Ky1.png

Not much different in terms of realism, but the person looks distinctly way more like Joe. Let's up the iteration number to 200.

https://pasteboard.co/ey0ZzC110CrK.png

We got a bit closer to what we wanted. Faces are a bit of a difficult thing to do, but i think we can figure it out. Overall, it feels a bit "wobbly". I noticed that it tends to be beneficial to decrease the CFG as you increase iteration number, if you want photos to be more photorealistic. Let's set it to 6, and the iteration at 200.

https://pasteboard.co/na1fH54LkqO2.png

I would say this looks pretty good, but I think we can do better. The important part is imo the prompt, and I think we can edit yours to get a bit better results. Here is the result for "portrait of Joe Biden in oval office, dslr" with 200 iterations, CFG at 6, and k_euler_a sampler.

https://pasteboard.co/noHbMgxhLU4s.png

That one was probably my favorite (or maybe it was the one before).

Overall, you can play with this almost infinitely. Adding different words to the prompt in different spots can yield pretty different results. And that's not even mentioning all the parameters one can tune.

that was a cool detailed breakdown.

how did you learn all that? i'm a noob at prompt engineering but would love to know how to get good at it with the least amount of time.

any good reasources to get up to speed?

Dall-e is kind of sad. It reminds me of Google Video, that thing before YT.

Casual users don't have a workflow to hook into, though. A website will be more convenient for them since there's nothing to install, and the web app probably runs faster than whatever hardware they're using.

I wish the same happened with GPT-3

The best thing about this imo is that it proved to me that these image generation algorithms aren't just regurgitating minor variations on an existing image somewhere in their database of billions of images. Although I understand the general idea of how these work, I still had my doubts. The astronaut one in particular made this stand out; many of these artists definitely did not draw astronauts, yet the resultant images do fairly well at emulating the distinct styles of the artists. Thank you for debunking one of my concerns!

I actually found 'the discovery of gravity' far more interesting. Depending on the style the interpretation was completely different.

I'm glad you noticed this too.

With "a horse", you always get a horse, but with something more abstract like "the discovery of gravity" or "a representation of anxiety", it's far more variable between a scene — perhaps people performing an experiment, or someone falling out of a window(!) — or it's simply a large block of text without any people in the image.

Yes! This is one of the most interesting aspects that I found. If it knows the particular style, it's typically very good at knowing how to use it.

I'm glad it could help clear things up for you.

I tried quite a few of them in stable diffusion, and while I found the results typically look worse, it was still interesting. How much curation of the output did you do or was it that exact prompt and the first result?

I would love to see it re-run with stable diffusion, and have those added along side/or ability to toggle.

I guess there are two scenarios:

1. The original prompt I gave it wasn't quite right, so I'd try a few different prompts with slightly different wording until I got something I was happy with. This happened with maybe 10% of the prompts I ran.

2. The prompt text was correct but it gave one or two bad images. In this case, I typically kept them rather than re-running them. I thought it would be a good showcase of its limitations. There's one particular type of image that I constantly (maybe 4 times) would get. It's so unique and I have no idea why it would generate that amongst other more "normal" images. You can see it here: https://generrated.com/prompts/extremeBokeh?prompt=extremeBo... — it's like a real estate listing. Quite strange.

This is fantastic! I know there's another platform https://openart.ai that is a search engine for dalle2 prompts. Could be a good complimentary to this.

This is fascinating.

Obviously it's handmade, but it gets me wondering -- is there any way to produce something like this algorithmically?

In other words, if you had full access to the DALL-E 2 (or Stable Diffusion or whatever) model... is there any kind of algorithm or method that could spit back the e.g. 100 most orthogonal text phrases that represent all the art styles?

I know that historically neural networks have been considered to be pretty much black boxes, but I'm curious if there's been any progress made at all in terms of interpreting them?

DALL-E tech aside, this is a great reference for artistic style.

I also think it’s really funny to have Sol LeWitt on there because he sort of pioneered the idea of translating text instructions to art. He’s probably the first person to explore the idea of diffusion through a neural network (art history experts feel free to correct me here).

Btw if anyone was curious I tried feeding some Sol LeWitt-style instructions to Dall-E 2.

It didn’t produce an accurate representation unsurprisingly although it might have made the connection to LeWitt

https://labs.openai.com/s/GSthErBUTNJMho6aXJnWf6Yu

Thank you! I actually ended up learning far more about particular artists than I expected. Either names I'd heard of but didn't know much about (https://generrated.com/prompts/edvardMunch), or brand new artists whose style I now love (https://generrated.com/prompts/okudaSanMiguel).

Is it? I only had time to go through the start of the list, but...

Accurate:

Berkey, Magritte

Inaccurate:

Arkley, Botticelli, Elvgren, Eng, Kahlo, and ironically Dali

Mediocre:

Bouguereau, Degas, Hopper, Kandinsky, Leighton

Oh! I guess this is a great reference for "what this prompt will do to a Dall-E picture, since it usually won't resemble the original artist that much."

Thank you for sharing this! I realized it was really hard to verbalize what I wanted from Stable Diffusion, so will be trying some of these and see what it comes up with.

Wonderful to hear — I really hoped it would provide some inspiration. It's great that our prompts can be (almost) anything, but it also makes it somewhat overwhelming to know where to start. I hope these can work as some starting points.

Wow, that's an impressive amount of images. How long did it take you to generate and curate so many?

Thank you! I think I first started to collect the generated images 2 months ago, with the help of some scripts I wrote — I have a list of prompts (around 350), a list of subjects (5 currently), and then I rename/check all of the saved images to find the ones that are missing.

Thankfully, DALL-E 2 includes the prompt text in the image filename when you download them. This was very helpful!

This is a phenomenal reference guide - thank you so much for creating it!

How much did it cost?

In total, I've spent around $500 with DALL•E 2 but I would say only half of that created images that went into the site.

Many of the prompts gave great results on the first try, but some of them required 5/10 attempts to get the prompt just right.

I've saved all of the generations I've created and at some point I'd love to upload them all to show how slightly different text can really change the images generated.

This is fantastic thanks for publishing it

That is really great! Thanks!

Reply on news.ycombinator.com