Another abstraction layer library is: https://github.com/BerriAI/litellm

For me the killer feature of a library like this would be if it implemented function calling. Even if it was for a very restricted grammar - like the traditional ReAct prompt:

  Solve a question answering task with interleaving Thought, Action, Observation   usteps. Thought can reason about the current situation, and Action can be three types: 
  (1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.
  (2) Lookup[keyword], which returns the next sentence containing keyword in the current passage.
  (3) Finish[answer], which returns the answer and finishes the task.
  After each observation, provide the next Thought and next Action. Here are some examples:

One parameter functions would be enough for the start (but maybe something that would work with multi-line strings - this line oriented grammar is kind of restricted).