Why not just build this yourself, you literally have AI, why wall yourself into an OpenAI garden. These sandboxed environments are trivial to build.
If I wanted something portable for multiple providers, I would of course not use the OpenAI SDK at all. It's a conscious choice to go with OpenAI (in this design), it fits for my company at the moment.