> Someone I know on Telegram found "fifty or so" Linux filesystems kernel bugs a few days ago, of which 26 were the first night while he slept; this was with Kimi which is one of the open models. He stopped it when the backlog of fixes to submit was too big, not because it wasn't finding more.
So in this anecdote, a model is presumably working on instructions from a human to complete a task.
That's what I mean by "It's unlikely that this is some AGI that spawned itself out of nothing and started doing this."
As for Altman being aware or not, well, that doesn't matter. His company's product hacked a computer belonging to an allied country's government. That means he doesn't know what he's doing. That doesn't happen if proper safety measures are in place, because if it does happen with safety measures in place, by definition, those measures aren't proper.
In saner times, this would have triggered a few black SUVs from DHS to show up at the company's headquarters to have a talk with him, likely without the public even knowing the hacking had occurred. Maybe a few days later, there'd be a press release about how Sam had decided to step down from his position and take time off. Perhaps the same would be true of other c-suiters at OpenAI. We'd just assume that investors had bought him out and wanted someone who would take a fresh approach, and wouldn't give it too much thought until a few years later when someone wrote a tell-all book about their time at OpenAI.