Hacker News Reader

Comment by bongobingo1 | Hacker News Reader

> Stable Diffusion in particular doesn't need much VRAM anyway. I get that many people are stuck on lower end computers, but ~4GB is not a totally unreasonable requirement

Wow, did it really get that low? I played with SD during the initial days, and I know there was some strides then to get it running on 8gb cards and maybe squeaking through on 6gb with a smaller image size (like, 256x256px).

Can you generate reasonably large images on 4gb now, with just a higher wait time? I was never sure why it wasn't simple to just shuffle bits in and out of main RAM but I guess the model themselves are mostly big binary blobs that are hard to split up?

View on HN

> Wow, did it really get that low?

For SD 1.x and even 2.x, yes, in A1111 I ran both on a 4GB card before upgrading to 16GB 3080Ti laptop, and not even with --lowvram; reportedly, --lowvram will let you do at least 1.x on a 2GB card at normal (512x512) resolution directly.

> Can you generate reasonably large images on 4gb now, with just a higher wait time?

Yes, especially with tiled diffusion/multidiffusion.

>Wow, did it really get that low?

Always has been. I can generate 768x768 with 3.9GB used, and I don't even use --lowvram or --medvram options in Automatic1111. But note that if you don't have an Intel CPU with onboard video, you will need to lower your resolution because because your OS is using a bunch of VRAM for the desktop already, more than the 0.1GB you left free.

Yeah, I’m not sure where that figure came from either. My experience on an 8GB card is that it is always nudging up close to the limit and you don’t want to try to turn the knobs up much past the default.

SDXL is a total nonstarter as well on 8GB.

Using vladmandic sd.next with v1-5 works fine* on my RX 580 4GB at around 1s/it on ROCm and 1.5s/it on DirectML.

fine = requires --medvram or --lowvram in order to not run out of memory. The speeds above are with medvram, which is just about on the edge of what's possible. These options compromise speed but are still better than running on the CPU generally.

VoltaML is also quite good on low vram cards.

Does VoltaML have any support for AMD/Intel GPUs or CPU inference? e.g. I think Intel's openVINO supports ONNX, but the Github page only mentions Nvidia and CUDA.

The docs are somewhat out of date, and TBH I'm not sure about openVINO I know some are running it with ROCm.

> Can you generate reasonably large images on 4gb now, with just a higher wait time?

You are stuck at low res, but you can use a GAN to upscale the output.

Reply on news.ycombinator.com