I’ve used llama.cpp easily for some local things, but it does lack a good ui.