Your mileage will certainly vary but I was able to eek out a lot more inference performance by exporting to ONNX, using a highspeed serving framework in that ecosystem and also relying on some computation graph optimizations that you can apply to the ONNX version using available community tools. Versus serving from PyTorch directly.

We were doing millions of inferences and we had a specific target of a couple thousand a second so a specific case for sure but that's my two cents.