Azure has access to your queries. Running locally really is the only way of having a privacy friendly LLM.
I wonder if there is some way you could do an E2EE LLM SaSS? Guess more work probably needs to be done into homomorphic encryption for that.
It feels like you don’t need encryption here, you just need a business model that doesn’t keep user accounts and sell the query data.
This would actually be a great use case for homomorphic encryption. I’m behind on what the state of the art is in that though, but my mind immediately went there.
From my engineering point of view, the state of the art of homomorphic encryption can do some maths very very slowly at a huge cost and can’t be used yet for any real use case. A very cool research topic, but it’s much simpler to run our software locally if you don’t want to leak your data.
It's insanely slow and probably wont be fast for anything for a long time.
I don’t think homomorphic encryption works like that
Well that was helpful.
Is the chatgpt api paid to openai not private to openai?