>run any model that talks to openai (which is nearly all of them)
What does that mean? I've never seen any locally run model talk to OpenAI, how and why would they? Do you mean running an inference server that provides an OpenAI-compatible API?
What does that mean? I've never seen any locally run model talk to OpenAI, how and why would they? Do you mean running an inference server that provides an OpenAI-compatible API?
So, if your model inference server understands the REST API spec that OpenAI created way back, you can use a huge range of libraries that in theory only "work" with OpenAI.
Worth clarifying that what the ecosystem/vendors have adopted is the "ChatCompletion" endpoint, which most models are under. But newer models (like codex) are only available under the Responses API, which the ecosystem/vendors haven't adopted as widely, AFAIK.