i run a home media server, can't wait to be able to add my own LLM service. it's just a matter of time for it to be something i can install over the weekend with proper hardware
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
./server -m models/7B/ggml-model.gguf -c 2048