Using a local LLM isn't a surefire solution unless you also restrict the app's permissions, but it's got to be better than using chatgpt.com. The question is: how much better?
An additional flavor to that: even if my professional AI agent license guarantees that my data won't be used to train generic models, etc., when a US court would make OpenAI reveal your data, they will, no matter where it is physically stored. That's kinda a loophole in law-making, as e.g., the EU increasingly requires data to be stored locally.
However, if one really wants control over the data, they might prefer to run everything in a local setup. Which is going to be way more complicated and expensive.
2. Small Language Models (SLMs). LLMs are generic. That's their whole point. No LLM-based solution needs all LLM's capabilities. And yet training and using the model, because of its sheer size, is expensive.
In the long run, it may be more viable to deploy and train one's own, much smaller model operating only on very specific training data. The tradeoff here is that you get a cheaper in use and more specialized tool, at the cost of up-front development and no easy way of upgrading a model when a new wave of LLMs is deployed.
Without a doubt. Companies like Mistral and Cohere (probably others too) will set up local LLMs for your organisation, in fact it's basically Cohere's main business model.
I am building something for myself now and local is first consideration, because, as most of us here, likely see the direction publicly facing LLMs are going. FWIW, it kinda sucks, because I started to really enjoy my sessions with 4o