If you want high performance inference us this: https://vllm.ai/

For local batch stuff: https://huggingface.co/docs/transformers/main_classes/pipeli...