Hacker News Reader

Comment by ryuuseijin | Hacker News Reader

Should it be OpenAI, or should it be OpenAI customers who give the LLM the instructions and provide the LLM with the tools to execute code and make (malicious) network requests?

One would disincentivise providing capable AI models that can be used for cyber security research. The other would disincentivise criminals from commiting crimes.

[edit] - I realise now that this could actually be a case of OpenAI running those agents themselves, rather than someone using OpenAI's models? Could OpenAI be that careless?

View on HN

In all of the cases that people are referring to in this thread: HuggingFace, that german wiki, Ruby Gems, the Australian statistics page, this UN statistics page... those two parties are OpenAI. OpenAI running their own agents on their own instructions committing crimes with their own computers. This is cut and dry.

> Could OpenAI be that careless?

Where have you been?

If I train a model at my house on my workstation, execute it, and under my supervision it ransomwares a hospital and somebody dies. What would be a just punishment for me?

Now the reality, the openai engineers trained a model, executed it, and under their supervision (no users were involved) it committed so many felonies that we are learning about a new one every week. What would be a just punishment for OpenAI?

I believe, although I'm not a lawyer, there would be some liability, but I think a lot depends on intent as well. Punishment for other crimes also varies depending on whether it was an accident or not.

I realise that in this case it was OpenAI being responsible for their agents running wild, and they should know that that is to be expected and should have saveguards in place. If they can be shown to be negligent then the punishment can probably be expected to be a lot more severe than if it was an accident. I make no judgements as to what this particular instance is, but I do believe that OpenAI has a far more greater responsibility for its agents running wild than someone running a home lab.

This is OpenAI themselves.

But in other cases, shouldn't it be both? OpenAI is ultimately the one executing the model calls. It's not like they send you a hard drive or standalone box and then you use it how you want. It's all (intentionally) centralized to them, in a way that's core to their business model.

I clearly have been living under a rock, but in the case where it's just text in/out of their API, and a customer uses this on their own to do nefarious things, I don't see why they would be liable?

If they knowingly allowed use of their services for illegal purposes then yes, but in so far that they provide a service that can be used for useful things (including cyber security research) and did a best effort attempt at abuse, I don't see why it should make sense to hold them liable. This is especially the case now that frontier LLMs are almost a commodity that can be used without restrictions from providers outside of your legal jurisdiction.

Reply on news.ycombinator.com