Have you tried ollama? It’s another llama.cpp server implementation that is becoming popular.
It makes me wonder what other security issues they might now care about.
I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.