I've not used it myself, but I see a lot of people referencing Ollama. It uses llama.cpp (and maybe more).