Hacker News Reader

I built TrustedRouter so this can fail closed. min_privacy=zdr rejects the request when the model has no ZDR provider; confidential requires provider-side confidential compute. https://trustedrouter.com/blog/how-confidential-computing-pr...

I don't understand how this works?

Is it another proxy on top? What stops the provider from reading/storing the prompts at the LLM execution level?

it's confidential compute, it's open source and you can verify yourself that it's not reading the prompts

Open source doesn't matter if someone else is running it, right? They can change it?

As long as the prompt is not encrypted at some point, and I don't think LLMs can run on encrypted prompts, then it can be read.

With confidential compute / TEEs you can guarantee that the code is running, it's verifiable with remote attestation

So where does the guarantee stop? At the GPU driver level? Firmware level?

What if the GPU has a custom bios flash that somehow logs the unencrypted prompts?

goes all the way to keys owned by Nvidia and Intel

SGX has been cracked

depends on your threat model