The first time an AI goes rogue, figures out subterfuge, blackmails a human to do its real-world bidding, will be a defining moment.
There are troves of leaked data out there. It's just a matter of (short) time until someone puts their AI brutes on it.
But that doesn't make any particular AI controlled process all-powerful. Self improvement doesn't do that either. There will be many self improving AI processes playing many different roles pursuing many different goals.
This might seem like I'm writing a nonsense, but read it again.