Having read the myriad AWS data protection agreements I would feel comfortable running bedrock hosted models. Others may feel differently.
What software do you use to run LLMs locally?
I tried Ollama but found requests a bit slow since it doesn't seem to fully utilize my resources. LM Studio has been better for me. It uses the upstream GGML implementation, which feels more optimized. My problem with LM Studio is that they've added so many options and features lately that it takes a lot of configuration.
What's your favorite open model for writing and coding these days?
I have used ollama, lmstudio, jan and vllm at different times, am readying for a wholesale transition to llamacpp.
I run Qwen3 locally for coding and writing. It’s a solid model.
Framework Desktop is becoming really popular. The one with the Max+ 395 and 128GB of RAM is an absolute beast. I might buy a Beelink GTR9 Pro (Max+ 395 with 128GB RAM), which costs around $2,000.
llama.cpp is the real deal. I’m using it as the engine for the product I’m building right now (https://tygra.ai/).
For work, I assume everything is used for corporate espionage, so it depends on the sensitivity of the data. If my employers / clients authorize a tool it becomes their problem.
For work, as an employee, sure, it's easy to say the company approved ChatGPT or Gemini, so you can go ahead and upload, for example, usage data to get a retention analysis. But what if you're the employer?
I assume I have no privacy these days. I've got Amazon sponsored spies in my house, listening at all times. I've got a completely insecure computer/operating system and an internet provider known to sniff traffic to sell info.
I've got a backup of everything on Backblaze's servers.. I assume the NSA has a copy of everything I've ever done... I joke they should just offer free backup service to everyone, and save some duplication. ;)
I don't have any secrets, I can't in the modern world.
This means I don't really have to worry, I'm free to just go about life online.
You can buy a good laptop and run open-source LLMs locally. 100% private. Disable its network. Stay offline.
Time to stop feeding the machine!
YOU CAN!
;)
The first thing to keep in mind is the illusion of transparency. You might internally know that something is wrong or exploitable or you've made an obvious mistake, but that's generally much less obvious to others.
The second to keep in mind is that we are currently in a crisis of attention. There's too much to think about and do nowadays, and there is a gigantic lack of motivated actors to act upon that information. You could consider it the dual of the illusion of transparency, but it's the illusion of motivation. Other people, by in large, just do not give a damn because they can't and don't have time for it.
Even a nation state if they wanted to go spy on everyone's private information would immediately find themselves with too much nonsense to sift through and not enough time to actually follow through even on surface level information. Let alone leaks that actually require some sort of sophisticated synthesis over two or three disparate pieces of info.
Lastly, it's the difficulty in exploitation. You know how projects and code and stuff seem easy until you try them, and it turns out that actually, this is taking forever, and it barely works? The whole devil in the details thing.
Well, that applies to exploits as well. It's easy until you try it, and then you have this Swiss cheese model of success where random stuff doesn't line up correctly and your workflow broke.
AI surveillance btw barely changes any of this calculus.
If you can run an open-source LLM locally on your own computer, completely offline, and use it for legal, finance, or medical topics, would you still say no to that?
I left this bit out because my original comment was getting long but I think it's important to be respectful over others' privacy wishes. So I didn't use an API when it concerns other people.
Of course, you need decent hardware to run LLMs locally, but you don’t need a super high-end computer to host qwen3:30b or gpt-oss:20b. You don’t even need a GPU for those models, as long as you’ve got a modern CPU. And they’re already pretty solid for writing and coding.
About that:
https://arstechnica.com/tech-policy/2025/08/openai-offers-20...
It says that The New York Times (NYT) is suing OpenAI for alleged copyright infringement, claiming that ChatGPT was trained on its articles.
What am I missing?
The core of the article. OpenAI is currently retaining all the chat data indefinitely - even for plus/pro users as s a result of a court order. Please read: https://openai.com/index/response-to-nyt-data-demands/
Sam Altman actually said this on a podcast with Theo Von recently.
“So if you go talk to ChatGPT about your most sensitive stuff and then there’s like a lawsuit or whatever, we could be required to produce that,” Sam told Theo.
He even asked Theo about his own ChatGPT usage, and Theo admitted he doesn’t use it much because of privacy concerns.
Sam’s response:
“I think it makes sense... to really want the privacy clarity before you use [ChatGPT] a lot — like the legal clarity.”
:O
But I'm not sure that in 3-5 years people will just give their sensitive financial, medical, or legal data to cloud AI providers without a second thought.
Code I only run through the zero-retention API accounts anyway.
Just last week I wanted to do some financial analysis with ChatGPT (GPT-5), but I wasn't comfortable uploading the CSV with all my transaction data. Instead, I used Qwen3 running locally on my MacBook to anonymize the data first, and then uploaded the sanitized version to ChatGPT.
¯\_(ツ)_/¯
"All these sites may as well have all my information." I don't think so. That's scary!
But you do have a choice.
If you want to upload your bank statements and ask an AI, "How did my spending habits change in Q2 compared to Q1?" or upload your blood test results and ask, “Give me specific nutrition, lifestyle, and supplementation recommendations," you can run LLMs locally. No data will leave your computer.
Personally, no I don’t send confidential data that I don’t want to leak online.
Personal: So, you download and run open-source LLMs on your computer?
Personally no I dont use LLMs for confidential data, the local ones that can run on my personal computers aren’t good enough.
Give qwen3:30b and gpt-oss:20b a try. You don't need a fancy GPU, just a modern CPU. Those models are already pretty solid for analyzing your personal data.
Investing quite a lot of time to figure out hosting LLMs locally in comparable quality without investing too much money (as smaller environments will never have a large budget for this).
Your sensitive data? load it up bruh