If an LLM figures out (or is trained to, whatever language you prefer) how to distribute itself, how would you ever go about cleaning that up? We're probably at least a few steps away from that currently, but how many?
Lets say you are an IT specialist in charge of a data center. Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems?
Because thats what it would take for AI to start taking over computer systems. Computers arent black boxes where some amount of correct input results in an exploit. Its totally possible to have a fully secure system.
Is this supposed to be hard? I setup a process that pipes input/output from the chat agent to a terminal and give it a prompt that contains whatever it needs to login.
That probably doesn't lead to anything too catastrophic happening today, but that's giving the chat box access to the data center.
> Its totally possible to have a fully secure system.
A lot of things are possible. It's a lot easier to have a less than fully secure system.
No, it isn't. As one example, the iPhone has been out nearly 20 years, with hundreds of the best security minds in the world trying to lock it down, and it still gets jailbroken.
This has nothing to do with the conversation of an AI hacking computers remotely through only data comm channels.
Or, without a melodramatically villainous human action in the mix, one which has been prompted to perform some innocuous task, but given inadequate safeguards, and for which it reasons (perhaps accurately, with the paramters it has been given) taking over every other computer will be useful in more efficiently carrying out.
So, I think there we be more chances for legit accidents, but also some motivated people intentionally trying to cause harm.