So MS invested $1bn into OpenAI. Meanwhile, these volunteers built an OSS version of OpenAI’s flagship product in their spare time.
Can you imagine if we could redirect funds to actual OSS developers at scale?
Can you imagine if we could redirect funds to actual OSS developers at scale?
Building a small version of something that already exits, has code you can look at (gpt2), paper etc. is going to always be cheaper and easier even if we ignore that OpenAI work on and publish more than gpt. This is before even mentioning that they have an enormous amount of free compute from Google (and others) while Azure credits is a lot of what the 1bn investment to OA included.
If there are volunteers who are capable of and want to contribute to the information commons and are merely constrained by compute, we should clearly help them.
(OK, I admit that it wouldn't have been possible without Google's donated TPUs. But the cost isn't so outrageous that it's out of reach for any reasonably funded startup).
> so a few dedicated nerds wanted to see how far we could get with that. In all honesty, we didn’t actually expect to get very far, but it was the height of a pandemic and we didn’t exactly have anything better to do.
Think about it: the pandemic caused many people to stop at the same time and for a long time. GPT-J took about 10 months to be released. It may not have been possible in a different context.
Plus, a lot of that $1bn was effectively an Azure compute grant (none of it was explicitly compute credits, but...)