Hacker News Reader

Really interesting. I wonder if you can do this on ollama too.

vLLM is simple to set up, use docker and make sure your backend (ubuntu or WSL ubuntu or whatever) has GPU support installed.