Hacker News Reader

Comment by boredemployee | Hacker News Reader

That's really cool! I always thought you needed a good amount of GPU VRAM to generate images using SD.

I wonder how fast would a consumer PC, with no GPU, generate an image with say 16gb of RAM?

On an Apple M1 with 16gig RAM, without using Pytorch compiled to take advantage of Metal, it could take 12mins to generate an image with a tweet-length prompt. With Metal, it takes less than 60 seconds.

And PyTorch on the M1 (without Metal) uses the fast AMX matrix multiplication units (through the Accelerate Framework). The matrix multiplication on the M1 is on par with ~10 threads/cores of Ryzen 5900X.

[1] https://github.com/danieldk/gemm-benchmark#example-results

Prompt length shouldn't influence creation time, at least it didn't in any of the implementations I used.

What is the resolution of your images and number of steps?

Defaults from the Huggingface repo, just copy-pasted. So, iirc 50 steps and the image is 512x512.

Edit: confirmed.

> Prompt length shouldn't influence creation time...

Yeah, checks out with my experience too. Longer prompts were truncated.

Some tools (e.g. Automatic1111) are able to feed in longer prompts, but then the prompt length does affect the speed of inference.

Albeit in 77 token increments.

Metal is such an advantage, had no idea

Wtf, my 4 year old, $400 crappy low wattage computer can generate a picture in a minute or two.

DDIM, 12 steps.

I was using a 6ish year old amd cpu with 16gigs of ram and generating a prompt would take about a half hour. Which is still massively impressive for what it is.

Use a free GPU from google colab and you can do the same in about 15 seconds...

Do you have a google colab link?

There is no shortage of google collab stable diffusion tutorials on the web

Which is why asking for one high quality starting point is such a useful question.

On 21 April 2023 Google blocked usage of Stable Diffusion with a free account on colab. You need a paid plan to use it.

Apparently there are ways around it, but I just switched to runpod.io. It's very cheap (around $0.80/h for a 4090 including storage) and having a real terminal is worth it.

yes, and if he does it on a paid machine with a better GPU it'll be even faster!

While true, neither your statement or mine above is germane to the discussion. It wasn't about how long it takes. It's a discussion of how cool it is that it can be done on that machine at all.

Reply on news.ycombinator.com