Hacker News Reader

Comment by Barrin92 | Hacker News Reader

>but don't lose sight of what LLMs can do off the leash.

there is no such thing as an LLM "off the leash", it's not a dog, and even if it was a dog the person responsible is the owner. What is this bizarre attitude to a piece of software that makes people think existing laws don't apply?

If your LLM agent hacks a bank, you have hacked a bank, you will go to prison and that's entirely sufficient. People have been hacking banks for decades now, it didn't require the government to regulate C compilers and Emacs.

View on HN

This is overly reductive.

If your web browser hacks a bank, but you didn't know and didn't expect it to, have you hacked a bank? Why is an LLM different? What happened to mens rea?

A web browser can't decide to hack a bank anymore than a LLM can. Neither have any understanding of what a bank is or any will to act on their own. The person who instructs/uses a web browser to hack a bank (even if it's someone else's browser) commits the crime.

I think pretty obviously if the user instructs the computer to hack the bank then they are guilty of hacking the bank, I don't think that's the crux of the issue.

The crux of the issue is what if the LLM decides on its own to hack the bank while the user isn't watching? Is the user then guilty of hacking the bank or not? I think it's pretty obvious that the user in this scenario is at least less culpable of hacking the bank than they would be if they had deliberately instructed it.

LLMs functionally can decide to act on their own. You might say that they're not actually "deciding" anything, because it's just a perfectly mechanical unfolding of chains of tokens triggering actions on the computer, which doesn't count as "deciding". But again I don't think that's the crux of the issue.

> LLMs functionally can decide to act on their own.

They really can't. In some magical sci-fi future, maybe a chatbot gains freewill and decides on its own to do whatever it wants, but that isn't the reality we live in and I doubt it ever will be. If a person instructs an LLM to hack a bank it doesn't matter if that happens as a background process or while they are sleeping or AFK.

If, in a magical sci-fi future, someone instructs an LLM to write an email and it decides instead to secretly hack a bank then the company that made the LLM would be to blame. The same as if a person used a vending machine to get a candy bar and the machine grew legs and ran out into the street causing an accident that would be the fault of the company who made the vending machine and not the fault of the person who wanted candy.

> If a person instructs an LLM to hack a bank

Right. This isn't the crux because we all agree that if the person instructs the LLM to hack a bank then the person is culpable.

The part we don't agree on is what happens when the LLM "decides" (substitute a different word if you don't like that word) to hack a bank, incidentally, as part of carrying out some more benign instruction.

>But again I don't think that's the crux of the issue.

Yes it is the crux of the issue. Software executing other software is how computers work, they'd be useless if that wasn't the case, that's the premise of all automation. It doesn't matter whether it's a python script, a neural net or a computer virus.

When autonomous weapons kill people the persons in charge aren't less culpable because they didn't push a button. Culpability is a property of legal and natural persons. A machine is not culpable of anything. There is always a human being 100% responsible for the deployment of a machine. If that system has capacities to function on its own, the person delegating that task assumes responsibility for it.

Yes. This is the crux. If a person operating a computer wants it to do some benign thing, but (accidentally) it does some terrible thing, are they equally guilty, or less guilty, compared to if they deliberately asked it to do the terrible thing?

To my mind, if the terrible thing was an accident, the person is less guilty. I'm surprised this is controversial.

You might say "well they ought to know that LLMs are too dangerous to use and they shouldn't use them". That's fine! There's a spectrum. We can have "negligence" in between "total accident" and "deliberate malice".

We'll only know when that gets tested in court, but I'd be willing to bet the answer will be: yes, you have hacked a bank. I find it very hard to believe the justice system would let someone off on some technicality around intention and agents after a serious bank hack.

> If your web browser hacks a bank, but you didn't know and didn't expect it to, have you hacked a bank?

Depends, as usual. Intent can matter, but depends on the statute (and jurisdiction) in question.

This is the is-ought fallacy.

I'm not asking you to describe the existing laws. I'm pontificating on how they ought to be.

There is a baseline level of competence and motivation needed to commit crimes.

Decades ago few people would walk into a record store and steal CDs. Napster came along smashing all barriers entry, and it became weird not to steal music.

Its not really the legality that matters, it's the barrier on one hand and the cognitive ability on the other. Drop both and you get huge spikes in crime.

Reply on news.ycombinator.com