Hacker News Reader

Comment by jdasdf | Hacker News Reader

I've been using v4 pro for the past few days and honestly in terms of quality it seems more or less on par with open AIs 5.4 or opus 4.6 (i havent tried 4.7)

To be clear, i'm not doing state of the art stuff. I mostly used it for frontend development since i'm not great at that and just need a decent looking prototype.

But for my purposes it's a perfectly good model, and the price is decent.

I can't wait for open model small enough for me to run locally come out though. I hate having to rely on someone elses machines (and getting all my data exfiltrated that way)

View on HN

You can use Tinfoil for inference, which lets you use the model in the cloud while getting similar privacy as running locally: https://tinfoil.sh/inference.

Disclaimer I'm the cofounder. This works by running the model inside a secure enclave (using NVIDIA confidential computing) and verifying the open source code running inside the enclave matches the runtime attestation. The docs walk you through the verification process: https://docs.tinfoil.sh/verification/verification-in-tinfoil

Hi there I use your service. It's great. But I have a few requests... Please support crypto payments...? Also you are missing some open source models (qwen 30b 3a, Deepseek 4 flash).

Unfortunately we don’t support crypto payments at this time as we use Stripe.

We try to add models selectively as we have to be mindful about our compute allocation. Is there a specific reason why you need those two models (and our models such as Kimi K2.6, GLM 5.1, Deepseek V4 Pro, Gemma 4 amongst others) don’t suffice for your use case?

Feel free to email me at tanya@tinfoil.sh and happy to continue the conversation there.

Tinfoil looks super interesting! Do you have load balancers in front of the trusted compute stack? Looked at a design like this in a different space and the options for ensuring privacy in a traditional "best practice" architecture seemed very limited

Yes we do, but the load balancer also runs inside the enclave and is attested: https://github.com/tinfoilsh/confidential-model-router

In turn, that attests the model enclaves, for instance, see https://github.com/tinfoilsh/confidential-deepseek-v4-pro. The model repo/release that the model router attests is included in the attestation config, which creates a chain of trust.

Also see https://docs.tinfoil.sh/verification/attestation-architectur...

Worth noting that NVIDIA confidential computing and similar schemes have been compromised and shouldn't be relied upon if it really matters. See https://tee.fail/ and similar.

I was interested in trusted execution environments and how safe they were. If you look on google scholar and start reading, they seem super vulnerable. The feeling is that the industry has no better option and that they are a way to tell customers they are safe when they're not

with physical access right?

While that does sound interesting, I don't see any benefit for me.

It would still ultimately exfiltrate the data outside of my control, and frankly i don't trust any "secure enclave" tech.

As far as i'm concerned physical access is root access, and for any private stuff that is wholly unacceptable.

Very reasonable if you have the resources to run it locally and certainly the best option.

But we created Tinfoil because not everyone has that capability especially when it comes to larger models, and it still doesn’t solve for the situation where you’re building a service for your end user and you want to lock yourself out of accessing their data. In those cases, this is the second best thing you can do.

The technical walkthrough section on this blog that we co-wrote with one of our customers walks through the various attack surfaces: https://www.workshoplabs.ai/blog/private-post-training

We weave in many mitigations against attacks, but it depends on what class of attack it is.

If there are specific attacks you are concerned about, happy to provide an answer if it’s something we can address or not.

Thanks for sharing your experience, I’m looking to try it out.

Which provider are you using for inference? Opencode or the DeepSeek api?

I just use the API directly. It's simple enough to setup and i like the control i get from just charging up and not having to worry about any random subscription taking money out of my account

Reply on news.ycombinator.com