Hacker News Reader

I'm considering switching my function calling requests from OpenAI's API to Mistral. Are they using similar formats? What's the easiest way to use Mistral? Is it by using Huggingface?

View on HN

easiest is probably with ollama [0]. I think the ollama API is OpenAI compatible.

[0]https://ollama.com/

Most inference servers are OpenAI-compatibile. Even the "official" llama-cpp server should work fine: https://github.com/ggerganov/llama.cpp/blob/master/examples/...

Ollama runs locally. What's the best option for calling the new Mixtral model on someone else's server programmatically?

Openrouter lists several options: https://openrouter.ai/models/mistralai/mixtral-8x22b

Reply on news.ycombinator.com