2s/iter? That is... Not great.
It is slower than CPU diffusion: https://github.com/rupeshs/fastsdcpu
Stable Diffusion in particular doesn't need much VRAM anyway. I get that many people are stuck on lower end computers, but ~4GB is not a totally unreasonable requirement.
VoltaML has some CPU optimized options for the torch.compile optimization.
Wow, did it really get that low? I played with SD during the initial days, and I know there was some strides then to get it running on 8gb cards and maybe squeaking through on 6gb with a smaller image size (like, 256x256px).
Can you generate reasonably large images on 4gb now, with just a higher wait time? I was never sure why it wasn't simple to just shuffle bits in and out of main RAM but I guess the model themselves are mostly big binary blobs that are hard to split up?
For SD 1.x and even 2.x, yes, in A1111 I ran both on a 4GB card before upgrading to 16GB 3080Ti laptop, and not even with --lowvram; reportedly, --lowvram will let you do at least 1.x on a 2GB card at normal (512x512) resolution directly.
> Can you generate reasonably large images on 4gb now, with just a higher wait time?
Yes, especially with tiled diffusion/multidiffusion.
Always has been. I can generate 768x768 with 3.9GB used, and I don't even use --lowvram or --medvram options in Automatic1111. But note that if you don't have an Intel CPU with onboard video, you will need to lower your resolution because because your OS is using a bunch of VRAM for the desktop already, more than the 0.1GB you left free.
SDXL is a total nonstarter as well on 8GB.
fine = requires --medvram or --lowvram in order to not run out of memory. The speeds above are with medvram, which is just about on the edge of what's possible. These options compromise speed but are still better than running on the CPU generally.
You are stuck at low res, but you can use a GAN to upscale the output.