Hacker News Reader

Comment by nikolayasdf123 | Hacker News Reader

LLAVA is pretty great

It is! Just downloaded it the other day and while far from perfect it's pretty neat. I uploaded a Gene Wilder/Charlie in the Chocolate Factory meme and it incorrectly told me that it was Johnny Depp. Close I guess! I run LLAVA and llama (among other models) using https://ollama.com

As a "web builder" I do think these tools will be very useful for accessibility (eventually), specifically generating descriptive alt tags for images.

Do you know of any good paid API providers offering LLAVA? I want to experiment with it out a bunch more without having to host it locally myself.

Cloudflare has it https://developers.cloudflare.com/workers-ai/models/llava-1....

Locally it's actually quite easy to setup. I've made an app https://recurse.chat/ which supports Llava 1.6. It takes a zero-config approach so you can just start chatting and the app downloads the model for you.

Just realized I read your blog about Llava llamafile which got me interested in local AI and made the app :)

What's your reservation about running it locally?

https://replicate.com/yorickvp/llava-13b :)

nope. I am self-hosting. support is pretty good actually. llama.cpp supports it (v1.6 too; and in openai API server as well). ollama supports it. open-web-ui chat too.

using it now on desktop (I am in China, so no OpenAI here) and in cloud cluster on project.

What are you using it for? Curious if there's any interesting purposes I haven't thought of

Reply on news.ycombinator.com