1. People who understand LLMs and know how to run them and have access to run them on the cloud. 2. People who understand LLMs well enough but don’t have access to cloud resources - but still have a decent MacBook Pro. Or maybe access to cloud resources is done via overly tight pipelines. 3. People who are interested in LLMs but don’t have enough technical chops/time to get things going with Llama CPP. 4. People who are fans of LLMs but can’t even install stuff in their computer.
This is clearly for #3 and it works well for that group of people. It could also be for #2 when they don’t want to spin up their own front end.
Being able to swap out models is also handy. This probably saved a couple of hours of my life, which I appreciate.
The fact it has configuration is good, as long as it has some defaults.
I'm more than capable of compiling/installing/running pretty much any software, but all I want is the ability to chat with a LLM of my choice without spending an afternoon tabbing back to a 30 step esoteric GitHub .md full of caveats, assumptions, and requiring dependencies to be installed and configured according to preferences I don't have.
Tl;DR It's for Mac users