There are plenty (perhaps far too many) tools for doing basically `curl` to OpenAI.
Local LLM tools are needed however; and are much better for deploying systems on terabytes of data at fractions of the cost.
Everything that goes through the tool can be logged to SQLite so this should make it easier to build up comparisons of different models.