Glad to see you're still working on something interesting. I saw the game console you developed many years ago. That paper demystified HDLs and the hardware side of the Z80 for me.
Simple API server for LLaVA based on llama.cpp. The llama.cpp project will add server and interactive support for LLaVA but I just couldn't wait. Runs great on my M1 Pro Macbook w/ 32gb RAM. Enjoy! :)
We already have ollama,LocalAI.
Neither of which run LLaVA as far as I can tell. Use this until that happens.
Also, this is way easier to set up. No dependencies. Easy.
why not? both uses llamacpp as backend
llama.cpp does not have interactive mode support for LLaVA nor the web server yet. So if they're just invoking the app, there is a lot of overhead for each call until they add that in. This server doesn't have that issue.