It appears to support the OpenAI API so you should be able to use it with offline servers that expose the same API such as llama.cpp etc.

Thanks for liking it. I will incorporate more LLMs.