Ask HN: How do you use AI for development in high security environments?

6 points·thesurlydev··4 comments

I'm curious how this is done effectively assuming no source should be sent to a model hosted remotely. Are there foundational model toggles for using inference and embedding but not using data for training?

View on HN

All the big providers offer no-training/retention guarantees (either by default, or as a toggle, or upon request). For many high security environments though I'd expect everything to be hosted on-prem or at minimum on company-controlled instances, which does limit your model options somewhat.

My employer has such contracts for some use cases, but actually forbids use of code completion/generation due to IP concerns.

I’ve worked with companies that would never trust publicly hosted models. But don’t have any issues with hosted models on AWS or Azure. But I work in cloud consulting so they already have to trust the cloud provider.

Yes this includes GovCloud implementations that have citizenship requirements and you can’t connect outside of the US.

I have not admittedly worked on any projects in the “secret” regions.

https://aws.amazon.com/federal/secret-cloud/

As someone else stated, there are enterprise services that offer solutions that make it so your company data isn't consumed, however, I think pretty soon we're going to see a lot of companies maintaining models locally in-house.

I think this is especially true given that Intel is shifting its focus toward an affordable in-house solution for training AI models locally with its upcoming GPUs.

Repeating what others have written based on my experience at the bank i work for- business offering will not use or save you data, and for more sensitive material we simply host it on prem

Reply on news.ycombinator.com