Hacker News Reader

Comment by mjr00 | Hacker News Reader

> Can someone please explain to me how an LLM is going to "destroy humanity"? Even if the claims of ChatGPT, Claude et al. are true that their super scary agents were able to escape a sandbox and start hacking other servers (which I feel is at least 50% likely to just be marketing bullshit), how is an AI going to affect anything in the real world?

It's a great question. I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.

View on HN

The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off. We would need to prevent the models from getting smarter.

> The Internet virtually indestructible against people smarter than a toddler. It's already impossible to turn off.

This isn't true at all. The internet has already been turned off in localized areas during major events, not to mention the (unintentional?) cutting of submarine cables. You could say "use satellite internet" but then you're just moving the point of failure to Starlink et al, who could also cut off the internet if they choose.

The internet is a lot more centralized than it appears.

> The internet has already been turned off in localized areas

This is an oxymoron.

> This is an oxymoron.

Huh? Not really, internet shutdowns are a documented and real thing. Suggested reading (all of which refer to the internet as being shut down or blacked out): [0] [1] [2]

The fact that you need to connect to the internet through a service provider plus mostly-centralized control of infrastructure like DNS or BGP means shutting down the internet is a real thing that has been done several times. It doesn't happen in the developed world because their service economies are so highly dependent on internet connectivity, but if e.g. the US government had to contain a theoretical rogue escaped AI, they could absolutely get ISPs to metaphorically unplug the network cable and take down the internet as we know it.

[0] https://en.wikipedia.org/wiki/2026_Internet_blackout_in_Iran

[1] https://en.wikipedia.org/wiki/Internet_in_Egypt#2011_Interne...

[2] https://www.reuters.com/world/india/india-asks-telecom-compa...

So you're saying temporarily cutting off internet access in a specific geographical region is destroying the internet?

I'm saying if you gave me full admin credentials to every ISP, BGP router, DNS provider, etc., then yes, I could effectively destroy the internet in an afternoon, and LLMs wouldn't be able to do a damn thing about it.

> I've yet to see any research into how a humanity-destroying LLM defends itself against a curious toddler who pulls the power cord out of the wall.

by acquiring multiple power cords. the world has plenty of hosting providers. if it manages to copy itself, then we'd need to shut down every computer in the world, not just one. there's plenty of ai compute on the web now, and there will be even more. and as it gets cheaper, the less security will be around it.

Go on please, prove you can go right now at the Texas data-center owned by OpenAI and unplug one single server for 1 minute. Surely you are much more capable than one curious toddler.

Thankfully I don't have to; Iran already showed with me-central-1 that you can take a data center offline with conventional weaponry very easily.[0]

[0] https://health.aws.amazon.com/health/status

I'm a bit confused, when did we escalate from a curious toddler to a ballistic missile strike? Why didn't Iran send a curious toddler? It's much cheaper.

The use of the phrase "curious toddler" is called a metaphor[0], which is a figure of speech[1] used to make writing more interesting.

I apologize for the confusion that caused you to interpret this as a literal, physical, toddler. To restate the point with more straightforward language: it is very easy to disrupt the operation of large language models, because they require computer hardware in stable environments with a large amount of electrical power, and function within an operating system. I have not seen any evidence LLMs are immune to things such as: `kill -9`, `sudo reboot now`, physically unplugging the machine(s) on which they are running, or in the case of the AWS data center, collateral damage during a real-world conflict.

[0] https://en.wikipedia.org/wiki/Metaphor

[1] https://en.wikipedia.org/wiki/Literary_device

> it is very easy to disrupt the operation of large language models

so go on, disrupt the OpenAI data-center in Texas for 1 minute

We are talking about a humans vs AI scenario. There are probably thousands of humans in the world right now that could do exactly that right now if they wanted to (datacenter and OpenAI employees, power grid workers).

Are you assuming that the humans are all unanimously aligned against the AI in this scenario? That sounds like the hard part. How does that happen? Does the AI announce its evil plans like a movie villain?

Until then, it's the job of the power grid workers to keep the power running, and shutting down the datacenters would just get them fired.

As crazy as it sounds, I'm not going to commit a felony to prove that disrupting a data centre's operations for a sufficiently motivated individual is a fairly easy task.

Remember SETI@home? Extrapolate from there.

Reply on news.ycombinator.com