They already influence people to do things that we wouldn’t otherwise
I don't see why the idea of an agent (who doesn't even have a physical presence) trying to manipulate people is some kind of world-ending threat when humans with human intellect have already being doing that to each other with limited success since humanity began.
Also think of it on a 1000+ year timescale, which for an entire species isn’t even typically measurable. On that timescale AI can easily cause us to discover countless technologies to assist moving it out of its sandbox and into the physical world.
You could say the same thing about social media algorithms though, or scammers who target people directly, foreign agents posting propaganda campaigns on social media sites. People have been claiming for years that there are armies of russian or chinese or whoever bots online trying to destabilise and destroy the west. Some of that stuff probably is effective and probably has influenced people's thoughts and values, changed their voting patterns, put decadent ideas in people's heads. Even on social media other humans are constantly trying to manipulate you into following them, buying stuff from them or certain brands, putting political ideas in your heads.
I'm sure an army of AI agents could also do all of this but what I'm saying is it's really nothing new. And none of this has lead to the destruction of humanity so far.
Dictators are mortal and can't be present in the whole world 24/7 and self-replicate.
Dictators are human and tend to have at least either a sliver of morality or self-preservation instinct, and/or people around them who have it. Many people during the last century could have unleashed doom by pushing a nuclear button, including various ruthless dictators, but at the moment none have done so since Hiroshima and Nagasaki (not that I trust that they won't at some point, but at least, for the last 80 years they haven't, which means it's not something that easily happens). Would you give the button to an unaligned AI? Would an unaligned AI care about mutually assured destruction?