Even small models can be a headache to export if they have even they use anything custom. I can't even imagine something the size of GPT-3.
Larger models aren't really more complicated than smaller ones though. GPT-2 is already supported, I believe only difference with GPT-3 is sparse attention.
The GPT-2 models included with Transformers can export to ONNX fine w/ a helper included with the Python onnxruntime.
To run in PyTorch the model architecture must be ported.
ONNX is slightly different; you could export the model to run in the onnxruntime but that has tradeoffs.