Hacker News Reader

Allegedly, based on info from a DALL-E 2 paper, it takes about 100k-200k GPU hours to train the model:

https://github.com/lucidrains/DALLE2-pytorch/issues/22#issue...

Assuming those are V100 GPUs at $3/hr, that’s at least $300k.

Relative to compensation for the developers and researchers working on projects like these this looks a small component.

GPT was rumored to cost in the millions, perhaps the hours estimate is conservative?

And presumably one should also factor in the cost of experimentation before a good setting for the final training is found.

How feasible is it to evaluate the quality of chosen parameters/settings without going through an extensive training? I mean <BIGNUM> GPU hours to discover your results are meh, jesus.

(Small timer with absolutely no idea of practical ML here)

4.6 millions

Reply on news.ycombinator.com