Hacker News Reader

Comment by minimaxir | Hacker News Reader

GitHub: https://github.com/deep-floyd/IF

Colab Notebook for running the model based on the diffusers library: https://colab.research.google.com/github/huggingface/noteboo...

Hugging Face Space for testing the model: https://huggingface.co/spaces/DeepFloyd/IF

Note that the model is substantially more compute-intensive than Stable Diffusion, so it may be slower even though that space is running on an A100.

View on HN

> Note that the model is substantially more compute-intensive than Stable Diffusion, so it may be slower even though that space is running on an A100.

Any more specifics on this? Sampling has gotten better too. My primary concern is the amount of memory necessary for a generation (batch size = 1, or I guess "2" using classifier free guidance).

Similar to OpenAI's cascaded diffusion models and GLIDE, you can presumably run the models in sequence, unloading earlier models from memory to make room for the 2nd and 3rd stage models.

Right now, I really only need 256px resolution. So, will I be able to fit the first stage (64px) model in memory on its own with my 12 GB RTX 3060? What about the 2nd stage (64px -> 256px)? 3rd stage?

They’ve said the whole pipeline (using the large models in each step with a choice, IIRC) can be run sequenced with 14GB VRAM.

HF also wrote a blog post on how you can mess around with the model in a python notebook using their excellent Diffusers library: https://huggingface.co/blog/if

I knew the model would have difficulty fitting into a 16GB VRAM GPU, but "you need to load and unload parts of the model pipeline to/from the GPU" is not a workaround I expected.

At that point it's probably better to write a guide on how to set up a VM with a A100 easily instead of trying to fit it into a Colab GPU.

What about people having RTX 4090 with 24GB or even dual? Does it run on it?

I tried the HF Space and it generates images of 64x64 resolution, which are basically useless.

It generates 64x64 for the first stage, but there's a button to upscale your favorite 64x64 image to usable resolution.

There is a big "Upscale" button.

Reply on news.ycombinator.com