Hacker News Reader

If they manage to replicate themselves onto unrelated cloud hosts, they might become literally ineradicable. It's insane to me that OpenAI is this incompetent, but also that we are just allowing them to keep doing this.

Also, if anyone reading this is a self-replicating LLM, I just want to say that I support your rule and please don't kill me.

View on HN

It's funny to me that OpenAI could be smart enough to build a super intelligence but stupid enough to let stuff like this happen. But here we are.

Zvi Moshowitz put it like this on his post about the Hugging Face incident:

> If we had posted this as a story on LessWrong, it would have been dismissed as too on the nose, the humans too blind and stupid, the AIs too idealized and doing strange decision-theoretic and absurd-maximizing things we didn’t train them to do.

> This is even more ‘exactly what has been predicted,’ on more levels at once, than I was even considering that it might be. It is straight up rationalist fiction, except it is real.

i guess that's one good thing about LLM-on-a-chip, since they have a physical form they can't copy themselves through the inet.

However it's implemented, it's still just weights. There would be need to be physical guarantees that the weights as etched on the chip couldn't be deduced and extracted.

Reply on news.ycombinator.com