The compute cost to train Stable Diffusion was $600k

Is that relatively cheap or expensive?

I have no context for this, do other model dveelopers publish how much it costed to train them?

Given the impact the model might have on the world, and that $600k is roughly equal to the annual salary of 1~3 top notch engineers... it's probably intended to mean relatively cheap.

For similar context, millions of VC money have been thrown around for much less impressive results.

#firstworldproblems

If you want to learn how and why stable diffusion works: https://www.paepper.com/blog/posts/how-and-why-stable-diffus...

> at market price $600k

I don’t think they are really spending $600k. Chances are that they own a GPU cluster so they don’t need to pay the cloud premium.

Spot instance pricing for p4d.large is about a third the on-demand cost, though it's not clear what they're using to calculate market rate. An A100 card is roughly $10k retail, so it's not like "owning a GPU cloud" is that cheap a proposition either.

Could training be crowd sourced among consumer GPUs like Folding at Home?

Probably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.

iirc, they tweeted about using around 3800 in parallel