Ash HN: How do you distribute or shard Stable Diffusion?

2 pointsthrwawy742 comments

A lot of HN has been having fun with stable diffusion. Do we really need 1 x GPU with 10GB of RAM? How do you distribute or "shard" a model you're training? Could we get this running on the raspberry pi clusters we all have? Hook it up to OpenFaaS too.

View on HN

I think part of my question is naive: The Raspberry Pi doesn't have certain hardware to make computations like matrix multiplication easy - certainly not as fast as CUDA cores.

However, I'd love it if someone could explain how one would conceptually approach dividing up the computation of a 512x512 image. Or if you had a well-trained model that happened to be 50GB in size, how you could distribute it over several compute cores (GPUs) like database sharding.

That is what this post is seeking. We need an AI shepherd :-)

As long as you want to create many small images (like 512x512) you can spread it over multiple GPU.

A friend is running cloud instances with up to 8x GPU and 8x stable infusion + api in parallel. He uses simple a simple round robin with a reverse proxy in front to spread the work.

AFAIK there is no feasible method to spread work on the same image over multiple GPU yet.

Reply on news.ycombinator.com