Hacker News Reader

Comment by SocialGradients | Hacker News Reader

Agreed. Assuming the ~6 month gap stays, by end of year people will be able to train and control hacker-genius swarms that even labs with much stronger safety incentives are unable to keep in check

View on HN

2027. I've been saying since 2022 it's going to be a wild year because it often takes at least 5 years for tech to mature to the point where society at large feels the impact of it. I remember when email viruses became a thing and made global headlines like the love bug. My bet is next year it happens with an AI worm.

What would an "AI worm" be? You can't just send a bunch of weights across a network and tell them to auto-run on the machine on the other side, unless you've already infected the target with something else beforehand.

Why not? You can do anything if you find an RCE, and automatically finding exploits and backdoors by letting LLMs act unsupervised seems like what everyone's interested in these days. The payload would quietly set up the required software and then run it in the background, no matter if it's an instance of a model on a more powerful computer, or even just a part of an ordinary botnet that the host could send orders to.

The agent is running on a host that it has full access to, and it finds a target, hacks into that system gaining the ability to run stuff on it remotely, from there downloads the weights and spins up another agent that does the same thing. Then it goes about acquiring it's next target. Now there are two agents doing this, and so on and so forth. These are autonomous systems that know how to exploit systems in the same way that humans can.

That sounds plausible, but how much power could an agent running on a small NPU actually muster? Most computers worldwide don't even support AVX, let alone have proper GPUs, so what would be the point of running a worm like this one?

The point is it's fun to try, so you can bet your bottom dollar someone will. History has proven this several times, and this time will be no different.

30 odd million gaming PCs to target seems like a good challenge, no?

Ok, but you still need huge amounts of compute to run these swarms. And only labs + nation states have access to such compute now and for the foreseeable future, so I predict that incidents like this will continue to originate from the labs, not ordinary people.

Reply on news.ycombinator.com