Show HN: a desktop app to discover, download, and run offline LLMs

Wow, this makes running LLMs locally significantly more approachable. Very cool!

Agreed!

This is great! CPU only for now or can it leverage GPU for sped up inference?

Focus is CPU for now - but it'd be very useful to have GPU support. As a baseline, this can support anything that llama.cpp. Relevant link: https://github.com/ggerganov/llama.cpp#blas-build

Looks very useful! Can't wait to try the Windows version. Thanks!

hopefully soon!

Follow up: Windows release is out.