Hacker News Reader

Comment by npsomaratna | Hacker News Reader

Data security and privacy. Our clients (in aviation, finance, etc.) need this due to legal and regulatory reasons. Also, the new Llama 2 models are very powerful. In my testing, Llama 2 70b is comparable to GPT-3.5 in capability.

(Shameless plug: here's our website: https://www.amw.ai/)

View on HN

I'm in the same boat. We have customers that need to run models in an environment that have access to the public internet, even if they did trust OpenAI et al.

More importantly for me, I don't want to be beholden to a model provider and have to take what they give me. I'd rather host my own model if an API was an option, because then I have control over it and can hack it as I want. I don't want to be just a wrapper on GPT which is sort of what you're stuck with if you just want to use their APIs.

Agreed. Worse, you don't want to be in a position where any single company can end your business at the flip of a switch.

I am also interested in your open-source-first approach, and am a bit confused from the landing page. Could you explain what exactly your product does? Is it an LLM-enhanced document parser?

have you considered azure's GPT, or is that not private enough?

This assumes you trust Microsoft.

We have. This is acceptable for some clients, but not for others. Both groups, however, prefer maintaining complete control over their data, given the chance.

Edit: plus, my personal view is that local LLMs are the future. They've already caught up to GPT-3.5 (based on my testing); and they continue to evolve rapidly. Makes sense to focus our limited resources on riding that wave.

OpenAI won't go away, but neither will they remain the first choice (or only choice!) for most use-cases.

There's some . . entrepreneurs . . who have been promising NIST/ITAR-compliant LLM frameworks on Azure, but when you ask around, they have not done all the legwork (AG/AGS). They're working off Azure Public, with "waivers" that they won't show anyone. Also, the history of their leadership is . . questionable. It all feels just a little hinky. Until that's cleared up, I advise anyone fooling with LLMs to do it on-prem, at least for the moment. One thing I'm worried about: doing LLMs with something like GovCloud is going to be absolutely bananas in terms of price-per-compute.

Looks great my friend

Reply on news.ycombinator.com