Hacker News Reader

Comment by digitaltrees | Hacker News Reader

Use your own mind and think from first principles. Can an AI agent execute a bash command to login to a web server? Yes. Can it call drop db? Yes. Can it provision a GPU and download model weights? Yes. Can it write an agile roadmap with a multi sprint plan? Yes. Can it follow that plan? Yes. Can all of that result in damage to core information infrastructure that is necessary for daily functioning society? Yes. Do humans descend into violence if there is food or energy insecurity. Yes.

What is missing from that to say AI safety is a reasonable position?

View on HN

What a silly comment. That's just the South Park underpants gnomes story with some extra steps. If there are vulnerabilities in food or energy production and distribution systems then those will eventually be found exploited by humans hackers regardless of whether LLMs are used or not.

What’s the legal mechanism for holding an LLM accountable for those actions? What’s the legal mechanism for holding human hackers accountable? Do you see the asymmetry?

Worse, you’re missing the entire point. Agents presently have the capability of doing society scale harm. It doesn’t matter if a human hacker initiates it or its fully autonomous, absent safety measures the harm is plausible. So hand wave away the rationality of safety measures but you haven’t actually shown why my point is invalid: AIs present abilities are sufficiently advanced to warrant safety measures.

What a silly comment. Agents have no capability of doing society scale harm so your entire argument is invalid.

Wtf are you talking about. An agent hacked the Australian Medicaid database. If it executed a drop db command that would be catastrophic. If it did something similar to a power grid or the financial system it could cause cascading failure across the economy.

Meh. Medical systems have been hacked many times before LLMs even existed, often by ransomware gangs. This sometimes delays elective treatments but hasn't been catastrophic.

Ultimately every security vulnerability will be exploited. Our best hope of preventing that is aggressive, unrestricted development and deployment of frontier LLMs for auditing and penetration testing.

Good not responding to any of my points.

Reply on news.ycombinator.com