Hacker News Reader

I (and I imagine many others) would love to use something like this, but can't, because my data is too sensitive to be uploaded to a cloud of which I have no gaurantees of privacy/security.

Is there any way we do this using rented GPUs and open-source software stacks? Paying for the service isn't the issue, I don't care if it's free or if a cut is taken in some capacity, I just don't want the provider to have access to my data.

View on HN

castform founder here. unfortunately, we are cloud-hosted at this point.

but some easy options on the open-source side include huggingface's trl & unsloth.

you can run our data-generation scripts here: https://github.com/castform-ai/benchmax and then hook it up trl/unsloth for training. should be able to do all of this on your own compute.

In OpenRouter there are Zero Retention options. And if you use a EU provider you can be somewhat sure that your privacy is given.

Other than that there is not really a difference to renting a GPU since the GPU provider can also just steal your data.

Local GPU(s) are always an option if you have the possibility. It is also not that difficult to run with stuff like “LocalAI”

Reply on news.ycombinator.com