Hacker News Reader

If you are curious about what use case there is of it: imagine that instead of JPEG compression, we are running an LLM. You send your prompt encrypted, server homomorphically executes the model and sends the encrypted output, which you then decrypt. As a result, you get LLM output, but the server operators never learn anything about the contents of the query or response.

View on HN

But.. if the LLM uses something like a MOE architecture, then surely some information about the prompt would leak just by observing what parts of the model are pulled into memory?

I think if you want full homomorphic encryption, you have to go for the worst case. So you always pull all the model? That's usually why FHE is not practical.

I wonder if FHE would ever be practical for an LLM. FHE necessitates calculating over the entire data, an extreme requirement for many workloads but routine for LLM inference.

Reply on news.ycombinator.com