If we were going slower maybe it would seem more theoretical. But there are multiple Manhattan-Project-level or (often, much) larger efforts ongoing to explicitly create software and robotic hardware that makes decisions and takes actions without any human in the loop.
We don't need some kind of 10000 IQ god intelligence if a glitch token causes the majority of the labor force to suddenly and collectively engage in sabotage.
If any of this can make decisions in a way that a human can, then I would start to question what human decision-making really amounts to.
Even the oracle version is already really dangerous in the wrong hands. A totalitarian government doesn't need to have someone listening to a few specific dissidents if they can have an AI transcriber write down every word of every phone conversation in the country, for example. And while it's certainly not error-proof, asking an LLM to do something like "go through this list of conversations and flag anything that sounds like anti-government sentiment" is going to get plenty of hits, too.
We already have widespread humans making their own decisions autonomously that aren’t in the best interest of humans, and we’re all still here.