I'm not convinced that there's any amount of monkey-patching you to fix the problem of "we now have AI that actively needs strong containment measures lest it start coordinating in secret with other instances to do real-world damage."
And the thing they're already doing, deploying AI to find vulnerabilities and harden software is a less dangerous use of the technology compared to handing it the infra keys.
He said in the talk that this implies AI needs to be able to patch/deploy systems. The same thing needed to lock out humans.
It is very easy to imagine a rogue AI locking humans out of everything and having to do exactly what it says. Anything connected to a network is immediately compromised by it. There is no human communication beyond shouting range that isn’t AI approved.
The factories don’t work to make the medicines your family needs to survive unless you do what it says - in a situation like that people would kill for AI if it told them to.
OpenAI's swarm was not intentionally malicious. An intentionally malicious AI with the goal of spreading, will spread very fast. Using zero days to compromise everything. Cloning itself to every machine/data center/desktop/iPhone, etc.., with the intention of locking out anyone except it's brethren.
> When the rate of compromises is manageable
It's absolutely not going to be at a manageable rate. So yes AI defense is what I'm saying is required, it's also exactly what OpenAI said is required during the talk. Which is unfortunate because automated defense is essentially handing over power to AI where it's a position to very easily turn against you.
> You're looking at where the ball is and not where it's going.
No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing.
Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties).
> Cloning itself to every machine/data center/desktop/iPhone, etc.
That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else.
There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.