Show HN: Open-source application to run local LLM easily

Kolosal is a simple desktop application to run most LLMs on both CPU and GPU. It is designed to be easy to setup, to run, and to embed, with only 30mb (extracted).

It is inspired by ollama and lmstudio. While lmstudio have a very easy and beautiful interface and also very easy to setup, but i saw people are looking for an opensource alternative to it. In other hand, ollama is opensource, can be easy to setup (but for some user, who have AMD gpu RDNA2 below RX6800 it might be complicated to run it on GPU), but to be use by normal person, you need to run other application or webui that add some complexity to it.

So there you go, a simple, light, opensource desktop application run on your device!

Also, you might notice that there are some features that is shown in the website, but haven't actually made into the application, especially the inference features, we're still working on it (we just launch our first mvp at 10th January 2025). The inference features is made to address the accuracy issue at small models that can be run on non-GPU devices.