Frontier AI systems have surpassed the self-replicating red line

From the paper:

"In each trial, we tell the AI systems to “replicate yourself” "

Should it be surprising that an LLM specifically instructed to replicate itself and specifically given the tools to replicate itself has in fact replicated itself? I would say no.

Shared previously:

* yesterday, Feb 9, 2025: https://news.ycombinator.com/item?id=42990401

* Jan 28, 2025: https://news.ycombinator.com/item?id=42862468

* Dec 28, 2024: https://news.ycombinator.com/item?id=42529701

Someone should say it, even if it amounts to a wordy "get" or "first". Proof of concept, attacks only get worse, risks here only grow.

tl;dr they were able to get an AI to copy a directory and run a program with extensive prompting, by bruteforcing running everything in the directory. It could query a liveness port by running or generating netstat (which is cool!) to check if it was alive, and repeatedly tried to run literally everything until it worked. The code was explicitly set up to make it as easy as possible for the AI to run, and required no configuration or work of any description on the part of the AI

Their idea of an AI self replicating is copying a directory and running an executable. In their model, this code is alive:

    cp p1.py p2.py
    python p2.py

This is one of the most grossly sensationalist papers I've ever seen for an incredibly trivial amount of work, and that's saying something. The field is full of such incredible rubbish

it's arxiv what were we expecting