Hacker News Reader

Comment by christoph | Hacker News Reader

One thing I’ve not seen mentioned/asked about DALL-E 2, which I’m really, really intrigued about is what hardware its typically running on and the average time it takes for image generation? Maybe I’ve somehow missed it, but I’m really intrigued by what their hardware set up is and how long it on average takes to generate the eight images from a users text input? I’m guessing it’s really quick, but I honestly have no idea… How does this system scale? Typical cost per image generated?

View on HN

Not a complete answer to your question but you may find this discussion interesting:

https://github.com/lucidrains/DALLE2-pytorch/discussions/10

Inference cost and scale seems to be much more favourable than large language models (for now).

In case anyone else is put off by the link referencing an answer that then links to something else with most likely higher hardware requirements that are not stated, the end of the rabbit hole seems to be here: https://github.com/openai/dalle-2-preview/issues/6#issuecomm...

TL;DR: A single NVidia A100 is most likely sufficient; with a lot of optimization and stepwise execution a single 3090 Ti might also be within the realm of possibility.

Allegedly, based on info from a DALL-E 2 paper, it takes about 100k-200k GPU hours to train the model:

https://github.com/lucidrains/DALLE2-pytorch/issues/22#issue...

Assuming those are V100 GPUs at $3/hr, that’s at least $300k.

Relative to compensation for the developers and researchers working on projects like these this looks a small component.

GPT was rumored to cost in the millions, perhaps the hours estimate is conservative?

And presumably one should also factor in the cost of experimentation before a good setting for the final training is found.

How feasible is it to evaluate the quality of chosen parameters/settings without going through an extensive training? I mean <BIGNUM> GPU hours to discover your results are meh, jesus.

(Small timer with absolutely no idea of practical ML here)

4.6 millions

Also very interested in this. AFAIK, the best alternative to DALLE-type generation is CLIP-Guided generation (such as Disco Diffusion [1] and MidJourney[2]) which can take anywhere from 1 - 20 minutes on an RTX A5000.

[1]: https://github.com/alembics/disco-diffusion

[2]: https://twitter.com/midjourney?t=-kKC5UE-gjIkMvAb709SyQ&s=09

I think it would be relatively quick.

The Latent_Diffusion_LAION_400M notebook generates six 512x512 images in about 45 seconds on a K80 on Collab.

DALL-E2 is more complicated but presumably also better optimised.

Reply on news.ycombinator.com