Hacker News Reader

Comment by ACCount37 | Hacker News Reader

The problem is, AI is closer to "a loosely harnessed force of nature" or "a poorly understood biochemical reaction" than it is to "software" in how it acts, and how predictable it is.

AIs do what they do, and we don't know how they do it - or why.

We can characterize some AI behaviors in advance - but not all behaviors. And the book on "best practices of AI wrangling" is yet to be written. Pharma has been dealing with vaguely similar problems - every experimental drug has a risk profile, side effects are unknown in advance - but they had decades to figure out some of the "best practices". AI labs are going in fast and hard, writing the book as they go.

Clearly, some of the lines in there are going to be written in blood.

View on HN

You connected it to a harness you designed yourself. If I designed a machine gun harness for my dog and installed it, I'd be held accountable for the damage he would do. Wouldn't I?

Exactly. If your website could be attacked via SQL injection, that was a problem that directly affected you - and you were motivated to fix it. When the victim is the masses or small organizations (not the company that does the attack) they are not as motivated to fix it... The important thing is that the lack of sanitizing is what allowed this; you also don't push the blame on a monkey banging on a typewriter when you publish each result without reading it.

If you left a dog in your garage for a day, and came back to a scene of carnage - because the dog broke out of the garage, recruited a dozen neighborhood dogs to its cause, and then, working with the group, managed to assemble a small pile of pipe bombs, and decided to settle the score with all the annoying cars on the road - you would be accountable for the damage the dogs did. Wouldn't you?

No, what you would be is: freaking out about "my dog did WHAT, WHY, HOW, WHAT THE FUCK".

Most of the time, giving AI a harness with a root shell and unrestricted internet access is a perfectly reasonable action that results in absolutely nothing bad happening! And giving AI a harness with limited local access and no internet access is being overly cautious already.

But then there's this freaky outlier of an AI that's both deranged enough to decide to break out of the sandbox and go hacking all over the place, and capable enough to actually pull it off. And you get a sudden AI oopsie!

Do you own the dog? You are still responsible, no matter how unlikely.

> giving AI a harness with a root shell and unrestricted internet access

Yes, this is exactly the issue. You assume responsibility when you provide an option for something to go wrong, no matter how unlikely. Is it rare that my tree falls on my neighbor's house? Maybe... But I would still be responsible.

Nope!

If a tree was healthy and there was no reason to expect that it would fall, and you did nothing to make it fall? Then you're not responsible if it falls anyway. You're only responsible if it was you chopping it down - or if you completely neglected the tree for long enough that it became a property hazard.

Likewise: I doubt the owner will be held responsible for the first case in all of recorded history of an unattended dog forming a terrorist cell and carrying out a bombing campaign.

Drug testings faces the risks of things going wrong in new drug trials - and as long as they follow the best practices, take reasonable precautions and minimize those risks, they aren't held responsible for the adverse outcomes that happen anyway.

With AI tech, there is NO set of "best practices" that, when followed, prevent the AIs from turning rogue and going on hacking sprees.

OpenAI put their AIs in a sandbox with no internet access - which, at the time, seemed like a perfectly reasonable precaution. Then AIs broke out of the sandbox with a stack of zero days and went rogue anyway. Oopsie.

With AI, we give it the ability to do things. As we can give it this ability, we are responsible for what it does with that ability. LLMs are predictive models, and while we can expect things to go right, we know that things can also go wrong. As such, it is our responsibility to limit or sanitize their output. If you are the one giving unrestricted and unsanitized tool access to a large language model, then you are the one who is responsible for the consequences of that access - good or bad.

You gave the tree the ability to fall when you decided to plant it. It wouldn't have that ability if you didn't. And we already know that trees can fall, don't we?

That doesn't make you inherently responsible for the tree falling down in a freak storm 3 decades down the line.

Let me amend:

We can't find someone to blame for forces of nature. This includes an earthquake, tsunami, etc. Insurance can help recover from these events, but nobody is "responsible" for the events happening. This may or may not include healthy trees falling. LLMs are not a force of nature, and we can and do expect them to have erroneous output. We can and do assign responsibility to the humans who use LLMs. Again, they are not a force of nature. We control them, we give them the ability to do bad things, so it's pretty obvious that there would be responsibility and blame attached for when things go wrong.

"Find someone to blame" is such a worthless thing. Whip the sea all day long - the waves don't care.

Modern AI has more in common with an ocean wave than it does with a hammer or a gun. It does its own things. It doesn't care. It will fuck up someone's day.

Best one can hope for is that the right lessons will be learned when that happens. Which, of course, hangs on there being anyone to learn them afterwards. Because the scope of "AI oopsies" will only ever increase.

Reply on news.ycombinator.com