OpenAI API Batch Requests

This seems like a fantastic approach to smoothing their GPU utilization.

It’s so difficult to compete with OpenAI - they ship so many “no brainer” features that are constantly setting the bar for what an LLM provider needs to provide - chat API, multi modality, function calling, json output, it’s endless.

Is running things in batch mode also cheaper? I don't see it, but it would be a nice way for OpenAI to smooth out their demand