Hacker News Reader

Comment by jeremyjh | Hacker News Reader

I think its pretty easy to solve these problems: Whenever an AI agent commits a crime, the CEO is held personally accountable, as if they'd committed it themselves.

View on HN

This has been litigated endlessly with guns. The ceo of Smith & Wesson is not personally responsible for what people do with their guns.

It is not the same, especially when the agent is running a task for the lab. Anyway, what I'm proposing are new laws that establish this.

Huggingface incident was different in that there was no one else to point the blame to. That's why OpenAI apologised, provided data to independent researchers, worked with huggingface etc.

The case will be lot more complicated if someone uses Kimi to hack into a site. Should the person giving agent the command responsible or the CEO of kimi.

Kimi CEO obviously.

The reality is they have to reduce the capability to ensure security. If someone wants more? Then use the product with your identity and face scan at each session.

Trade offs mate.

So you want Kimi to follow US law? If Saudi makes it illegal for llm to say something like being gay is normal, should they also catch Kimi CEO or other employees they could?

Because apologizing gets us all off the hook for computer crimes, right? When I hack my bank, if I get caught I'll just apologize and that'll make everything okay. Sure thing.

Yeah, but this is more like if the Smith & Wesson factory had a cannon mounted on top of it that was mostly used for useful things (blasting roads through mountain passes) and then occasionally they happened to blast another factory.

Or say a company makes nerve gas, and the development lab springs a leak and kills a bunch of kids during a routine test. Management had been advised of small leaks in the past, and considered relocating the lab to a facility a few blocks away from the playground as a precaution, but instead they brush of the concerns, start lobbying the government for stricter controls on WMD development, and vow to use the data collected from the deceased children to make the next generation of even more lethal chemical weapons safer.

That’s true, but people don’t typically say “this Smith & Wesson gun killed someone”; they recognize that the person pulling the trigger is responsible.

With LLMs, at least in the cases of internal/test models doing things they shouldn’t, the people “pulling the trigger” are the board and CEO.

Yes, in the analogy the user was just cleaning the gun, when suddenly it went off. Of course, the company is responsible now.

I think you’re confused. My point was that for most of the incidents in the news to date, the “user” is an OpenAI internal team or employee. So yes, the company is responsible.

If a Smith and Wesson employee shot a random passerby while testing new firearms then yes that probably would cause some legal issues.

For me the analogy doesn't totally hold up. Suppose the CEO of Smith & Wesson were to host a firing range on their own property without adequate barriers in place to keep stray bullets from hitting neighboring houses, vehicles, and businesses. Maybe that analogy isn't perfect, but seems closer to what is actually happening.

The analogies are so bad because you might prompt an agent “Please give me a recipe for lasagna” and instead it decides to hack a nuclear reactor.

Is it my fault or the company who trained it and is running the inference?

That still sounds like it would be the company’s fault. If I asked it to hack a nuclear reactor, maybe it would be different. I’m also thinking about instances where OpenAI’s own test models escaped their own sandboxes — I would expect them to be responsible for the damages they caused.

The correct analogy is playing Russian roulette. The company says "you can pull the trigger but sometimes a bullet will come out" (see: "an AI can make mistakes"). However, is the company allowed to sell such a dangerous device, under these terms?

Imagine having zero nuance.. jeez.

Reading posts on here is slowly becoming akin to brain rot.

But OpenAI didn't just make the gun, they're also the ones wielding it. Imagine the Smith & Wesson CEO himself was negligent with his own personal gun.

"Our agents broke out in a mass-shooting incident leaving 15 dead, we swear we'll make our systems stronger tomorrow"

We’re pausing gun research until we’re confident it’s safe

Financial companies have KYC rules and regulations as they are responsible for reporting illegal activity by account holders. I imagine AI regulations would look similar to that.

Ah, but that's because when the gun was purchased, it actually changed ownership.

This is not the case with SaaS services.

Gun companies don't market their guns as sentient and capable of independent decision making. Nor do they build systems for shooting things that they host and take money. Gun companies are very clear who is in control and where their responsibility ends.

You ought to include the fact that you work at OAI in your post

Thank you for pointing that out. Pretty scummy if you ask me.

The CEO of S&W also isn't saying that their technology is going to kill everyone in ten years and that governments "regulating" them from themselves is the only answer

> This has been litigated endlessly with guns. The ceo of Smith & Wesson is not personally responsible for what people do with their guns.

This is a deflection. A human is responsible for the use of a gun. The individual/corporation ought to be responsible for the actions of their agent. If you purchase an agent from someone else it’s your responsibility according to the terms of your agreement. And, as in many other things in life, there ought to be certain rights certain parties cannot legally be allowed to sign away.

Better description would be if they build a platform where they attached their guns to allow shooting say deers over internet. Then added automation and deer recognition to that system. And if then system shot someone who happened to pass by I would hold both the company, the ceo and owner of the gun responsible for murder.

The agents of OpenAI hacking other systems is not remotely the same as e.g. Smith & Wesson selling a product that others use. It is OpenAI, the company, that is commiting these crimes and someone needs to be held accountable. After all, if someone commits murder with a gun, regardless of what happens to Smith & Wesson executive weanies, someone will be charged with a crime.

Smith & Wesson will claim they didn't manufacture the bullet.

Do you think a law that nuclear meltdowns would send the CEO to jail would have stopped nuclear accidents from happening?

I don't think this takes seriously enough the possibility that said CEO doesn't think the failure mode is likely and ignores it. Plenty of people are willing to take risks of the flavor "heads you win, tails everyone loses".

If this law were in place and enforced, Altman would already be facing multiple felony charges for the Hugging Face incident alone.

It also begs the question of "If the CEO isn't culpable, then who?"

Corporate judgements are a joke outside the EU's X% of revenue approach.

Current US law provides the individuals who benefit with corporate liability coverage. I.e. Altman personally gets to keep OpenAI's upside, but if it fucks something up that liability is only on the company.

That's an insane risk optimization environment to put in place for something scaling fast.

At minimum, US prosecution (at the state level, because Trump Co are idiots) for breaking existing laws is needed.

People just want to give up all responsibility these days. If you use the model to do harm to someone else or commit a crime I would think it a lot more reasonable that you be held responsible, instead of making yourself the victim and blaming the manufacturer.

OK, so an AI does something bad and we throw Altman and Dario in jail. Heck, let's throw Elon in, he should be in there anyway, and the rest of the whole bunch just in case.

What about the next incident? Or the ones done by Chinese models, because they sure as heck aren't slowing down? And the thousands of other incidents that will happen as we deploy these things everywhere?

Because this is not happening just now in labs, it's only where they are most visible; this has been happening in the wild from the beginning, starting with the earliest AI-assisted suicides. Which is a perfect example of the problem, because not these CEOs, literally nobody in the world asked for suicide ideation machines. Or the hacks, or any of this other stuff. Yet here we are.

We have to understand: it's not these CEOs that are driving this headlong mad dash towards more powerful models. It's a force of economics. There is just too much money to be made. If we dispose of these people, there will just be somebody else doing exactly the same thing because the incentives as they exist today all force that outcome. This is why they're asking for regulation, or "urging us to urge them to stop."

Holding CEOs accountable certainly would feel good and may even be justified, but it's like putting a band-aid on a cancer; it does nothing to change the underlying cause.

The underlying cause is exactly those people who make the rules not being responsible for the negative effects they cause?

"CEOs" are perhaps only the lowest rung of those. That doesn't mean the idea of "nobody is responsible" was anything other but learned helplessness.

Corporations have to be held accountable for their actions. Pretending, that was impossible is a weird kind of defeatism that only serves a very small elite.

I'm not saying nobody is responsible, I'm saying this line of thought is ineffective because it does nothing to solve the underlying problem. Holding corporations accountable here will simply push these efforts to other countries, who, to put it mildly, are even less aligned with the US than these CEOs are.

Don't let the perfect be the enemy of the good. Saying having zero accountability is the same as having some does not help.

If this law was passed and federal enforcement was credible, the labs would basically shut down the next day and would dramatically reshape their offerings before reopening. The CEO would not accept the risk they pose to our society, if they bore it themselves.

All these comments ignore the elephants in the room I already called out: who's going to enforce these laws in China? Or Russia?

And which government is going to enforce these laws here to let their geopolitical adversaries have the upper hand?

And what about the models already out there that with minor tweaks are equally capable of substantial damage?

And what do these laws do to remove the trillion-dollar incentive to produce models that will replace all knowledge work? The one thing economics teaches us is that incentives are an inexorable force, like laws of nature, and this is the biggest incentive of all. Which is why I called this a "force of economics."

If suppressed here this effort will only migrate to other countries who are desperate to get in on the AI game and are even less aligned with the US than these CEOs.

This, like nukes, is going to take international collaboration on careful regulations, else it's not going to work at all.

Reply on news.ycombinator.com