> The whole training process took about
7/14 days on a cluster of 16/32 nodes for 1.5B/7B model, each equipped with 8 Nvidia A100
(40GB) GPUs.
The former CEO of Stability estimated the Dall-E 2 training run cost as about $1MM: https://x.com/EMostaque/status/1547183120629342214
It is so nice to see that you don't need tech oligarch level of compute for stuff like this.
So yeah, I imagine this is not a big deal for large, well funded, universities.
Biggest issue with these is ROI (obviously not real ROI) as GPUs have been progressing so fast recently for AI usecases that unless you are running them 24/7 what's the point of having them onprem.