Hacker News Reader

Comment by wokwokwok | Hacker News Reader

Is it self hosted?

Yes => secure.

No => not secure.

QED.

Unless you have the power to define your own information boundaries (eg. via your own security in your own cloud account, or your own network), you can't assume privacy is protected.

Any kind of privacy assurance is only as good as the word of the salesman.

Would you send your medical records to an arbitrary 3rd party? Would you send your production code base to an arbitrary 3rd party?

There are very few entities that people trust for this (eg. AWS) because they have a history, reputation and audit history to support their claims.

View on HN

I agree. But there are hardware based solutions called secure enclaves that enables software companies hosted on AWS (like us) to serve a SaaS to users without technically seeing the data. This can be verified remotely even before sending data to us with attestation.

This is called AWS Nitro Enclave and this is one of the hardware we will cover. You can find more about it here: https://aws.amazon.com/ec2/nitro/nitro-enclaves/

Ok, I'll bite. I'm happy to be wrong if I am wrong...

This document [1] defines a Nitro Enclave as an application image that runs in a secure runtime with no external networking.

This document [2] shows how you build a secure image. You basically take an image (like [3]) and run `nitro-cli build-enclave ...`.

Ok... so explain to me:

- Given that the image you build has to contain 1) private data (mine) and 2) private models (yours).

If I build the container image, how do you protect your models? (because I did, to access the base image)

If you build the container image, how do I know you didn't access my data? (because you did, to build the image)

What am I not getting?

[1] - https://docs.aws.amazon.com/enclaves/latest/user/nitro-encla...

[2] - https://docs.aws.amazon.com/enclaves/latest/user/getting-sta...

[3] - https://github.com/aws/aws-nitro-enclaves-cli/blob/main/exam...

I don't think that the model they are using is private. You can download it[1] and run yourself. Looks like it is GPT-NeoX which was fine tuned using the open source LAION OIG dataset[2].

[1] https://huggingface.co/togethercomputer/GPT-NeoXT-Chat-Base-...

[2] https://huggingface.co/datasets/laion/OIG

If I can build the nitro image myself, I would just self host it.

If this is “self hosted ML” then that’s great, and it is secure!

…

That obviously isn’t their business model.

(Or is it? Did I totally misunderstand the offering?)

    import blindai
    blindai.api.Completion.complete("I love AI…

...I dunno, I feel like they'll say something like:

- You have a client side encryption key

- You encrypt the data before you submit it

- The encrypted data is put on the image along with our Magic Sauce

- In the secure enclave, the data in encrypted and processed

- The result is encrypted

- The encrypted result is returned to you

- You decrypt it locally!

The problem with that is there's a whole lot of magic hand waving about how the secure key that theoretically never leaves your local side, ends up inside the container image without them having access to it.

How does it actually work?

The client side SDK we provide manage the security for you. As the client side is open-source, as well as the server you can verify the claims I will make. In those simple lines of code we:

- The enclave uses primitives from AWS that we cannot fake to create a certificate containing a hash of the code loaded, a public key to exchange keys, and other security information. This certificate is signed by a hardware derived key from AWS, and therefore we cannot forge such certificates. Here I talk about AWS but it can be Intel or AMD depending on the solution you choose.

- You receive the certificate, check it's valid locally using AWS public key.

- Once you know locally on your machine that we are using a secure enclave without backdoor that will handle your data properly (because it is a valid enclave and you can check that the hash you see in the certificate is the same as the audited open-source code), you can finish setting up the TLS channel using the public key inside the certificate

- Data is encrypted locally and sent through this TLS to the enclave

- Data is decrypted there, where we cannot peek due to isolation

- AI model is applied inside

- Output is encrypted inside enclave

- Output is sent back to you for you to decrypt it

I hope it makes it clear!

Reply on news.ycombinator.com