This. LLMs don't need to be conscious to wreak havoc. The moment they reach the proficiency level of a junior programmer we're going to have a massive shitshow because so much of what we do is online and so much of that stuff is not written/maintained correctly.

It's only a matter of time before a rogue AI erases a bank's database or randomly triggers an anti aircraft missile or something similar.