ollama is another good choice for this purpose. it's essentially a wrapper around llamacpp that adds easy downloading and management of running instances. it's great! also works on linux!
Luckily llama.cpp has come a long way and was at a point that I could easily recommend as the open source option instead.