Aren't there already much more efficient ways to make inference private?
Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/
Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/
Heh yes absolutely, but there is some nuance.
Secure Enclave still requires you to trust the operator and also trust that it’s configured properly, supply chain is secure, etc.
The beauty of FHE is that it doesn’t rely on the compute being secure. All you need to secure are things you already have control over as a client.
I agree with you it’s still way too slow to be generally useful. (By general, I mean practical for arbitrary computation — you can relax the requirement and have fast homomorphic encryption if you only do specific kinds of operations).
Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.
the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Intel/AMD's promise that their hardware is bugless/backdoorless, and that your adversary cannot directly inspect bits in the hardware
- The output can reveal information to the provider, which homomorphic encryption would have protected
- Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit
So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.
That's why they provide cryptographic attestation that the open model they're running is exactly what they advertise without any modifications.
That combined with GPU confidential compute should protect your LLM prompt and output.
E2E encryption (including homomorphic encryption) have the nice property that there are much fewer ways for things to fail.
(Tangentially, attestation is basically trying to ensure that faults are obvious, but that doesn’t reduce the probability of the faults in the first place).
1. A ZDR clause is "trust me bro". You have zero way of verifying their pinky-promise.
2. A ZDR clause is still subject to the old-classic "government, court or administrative order" catch-all clause. :)
3. "Even with ZDR enabled, Anthropic may retain data where required by law or to address Usage Policy violations. If a session is flagged for a policy violation, Anthropic may retain the associated inputs and outputs for up to 2 years, consistent with Anthropic’s standard ZDR policy." (I quoted Anthropic, I'm sure all the others have similar).