HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by beebaween | Hacker News Reader
Full thread
beebaween
·
What's the best way to run this is I prefer to use local GPUs?
View on HN
fzysingularity
·
We’re adding this as we speak. Ollama support is already there, and here’s vLLM inference:
https://github.com/vlm-run/vlmrun-hub/pull/120
EarlyOom
·
You can try out some of our schemas with Ollama if you want:
https://github.com/vlm-run/vlmrun-hub
(instructions in Readme)
Reply on news.ycombinator.com