Does anyone know if they'll ever release the models for GPT-3 so we can train/re-train then ourselves? Or is GPT-3 so general that it doesn't need retraining?
I don’t know if they will release the models, but are you sure you can train a 170 billion parameter model? Last I heard it’s around 500GB, which would require serious infrastructure.
What's interesting with machine learning is that in a few years time algorithms get efficient enough to train the same quality models on commodity hardware. At the same time organizations are always a few years ahead :(