Comment by simoncion | Hacker News Reader

Comment by simoncion | Hacker News Reader

Parent Full thread

> Because we continue to have zero evidence that aligment is an actual risk.

I disagree. Every time one of these LLMs -say- interprets an attacker's instructions as either its system instructions or those of its user, interprets its own internal chatter as a user's command to perform a destructive operation on that user's data [0], burns all of the user's budget from getting stuck in an incredibly stupid loop, massively overbills the user because it can't reliably report which system the user is using [1], encourages a user to swap their usual cooking salt for sodium bromide, etc, etc, etc, that's a harmful alignment failure.

These are real harms happening right now due to alignment failures. They're just not harms to the future of the entire species... what doomers call "existential risks", or "x-risks". You'd think that the fact that these machines are so amazingly unreliable would be a large part of the "x-risk" conversation, but... well, it makes sense that folks like writing speculative science fiction much more than they like doing investigative reporting.

[0] This general problem happens a lot, but I'm specifically thinking of that one where the Claude LLM's internal chatter lead it to believe that the task it just started was done, so it instructed the Cloud Provider to destroy the mess of "AI"-GPU-attached VMs... along with a bunch of very-expensive-to-produce data from the in-progress run.

[1] <https://github.com/anthropics/claude-code/issues/73597>

> These are real harms happening right now due to alignment failures. They're just not harms to the future of the entire species

Okay, sure. You can also cut your hand off with a chainsaw. Everything you describe seems amply solvable with existing tort and liability law.

Customers are willingly entering into business with OpenAI. I don't see an argument for preventing OpenAI from "building these systems" just because their products are buggy.

> Okay, sure. You can also cut your hand off with a chainsaw.

No, the correct analogy is one where the major LLM providers are selling cars intended for use on US interstate highways and other public-access roads, but have designed and built these cars with the very latest in 1940's safety systems and construction. Featuring innovations such as "Our rigid solid steel construction means the occupant is the crumple zone!", "You'll love the crushed heart and jaw our steering column delivers!", and "Your passengers will enjoy picking glass out of their faces for the rest of their lives when they're ejected from the cabin's open bench seating through the plate glass windshield!", it's a car that will be sure to wow the market.

Well... it would wow the market, except that -in the US, at least- it's illegal to sell a new car intended for use on public roads that ignores the last seventy five+ years of automobile safety lessons we've painfully learned.

"Differentiate between data you know comes from sources you control, data you know you have thoroughly sanitized, and unsanitized data that comes from an untrusted source, or else attackers will gain control of your system." is something that you can't get a CS degree without understanding, and can't be in the industry for more than a few years without encountering repeatedly. We're not talking about designing new cryptosystems... we're talking about "Don't blindly trust everything you're told by strangers.". You don't even need a CS degree to understand that rule.

> the correct analogy is one where the major LLM providers are selling cars intended for use on US interstate highways and other public-access roads, but have designed and built these cars with the very latest in 1940's safety systems and construction

Sure! I'm not defending these fuckwits. I'm saying their form of harm isn't novel.

We don't need new legislation to prosecute and litigate. We just need to enforce the laws on hand. I'm halfway convinced the arguments that this is all novel voodoo are for both fundraising and liability mitigation.

> I'm saying their form of harm isn't novel.

Your initial attempt to brush off my comments about how -contrary to their assertions that they're extremely concerned about safety- these LLM companies produce products that very, very often cause harm due to "misalignment" caused -in large part- by ignoring basic data-handling lessons we've learned over the past like thirty years with "Okay sure. You can also cut off your hand with a chainsaw." indicates your lack of understanding of my point.

> We just need to enforce the laws on hand.

What laws? Be specific.

Keep in mind the generally-low quality of both Microsoft Windows and much-to-most commercially sold software, [0] as well as the fact that -in the US, at least- it's currently totally legal for companies to sell such shitty software, just so long as they don't substantially misrepresent what it can do and trigger "fraudulent claims about the product" consumer protection laws.

[0] ...SaaS or otherwise...

> indicates your lack of understanding of my point

Sure. Help me understand. I've sat in policy circles and partaken in the hysteria, and now I'm reversing on that initial trust in AI zealots convinced what they're building is magic.

> What laws? Be specific

Liability. Tort. The ones making their way through courts around e.g. ChatGPT killing kids.

> Keep in mind the generally-low quality of both Microsoft Windows and much-to-most commercially sold software

Has anyone alleged Windows killed a kid in court? If not, not comparable.

> Help me understand.

Okay. Read these comment threads, then tell me if they change your understanding of what I've been talking about in this thread: [0][1]

I'll address the rest of your comment after you get back to me.

[0] <https://news.ycombinator.com/item?id=48999644>

[1] <https://news.ycombinator.com/item?id=48999415> [2]

[2] Yes, I'm aware that that one is the one we're talking in right now. You should re-read it with both the context from [0] in mind, as well as your initial comment to which [1] is a direct reply to, namely:

  > Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right?
  Because we continue to have zero evidence that aligment is an actual risk.

Thank you, the "LLMs can do no wrong" bunch is ab exceptionally odd take from my point of view. LLMs are already causing all kinds of social issues, and the evidence of this exists in massive amounts. At least to me living in the US and the sue happy culture we have here, how much said AI providers have gotten away with so far surprises me.

> the "LLMs can do no wrong" bunch is ab exceptionally odd take from my point of view

It's also a take nobody has made.

To be fair, there are really three threats:

a) People do bad stuff because LLM told them a wrong thing. Example: AI told me I should treat my heart attack by putting a fork in the outlet. Maybe similar to seeking medical advice on reddit?

b) People use LLM to do bad stuff. Example: People use LLMs to find 0 days. Get cooking recipes for poison. Write better phishing letters. This has parallels to the gun legislation question.

c) LLMs do bad stuff on their own, beyond what the people that use it intended. The case at hand might be an example of this. Maybe similar to having an animal as a pet. We will see if it's more like a house cat, lion, or black plague.

Reply on news.ycombinator.com