Initial testing has only been done with ollama. Plan on testing out llama.cpp and vllm when there is enough time