The hypotheticals explored in the article linked by upwardbound don't deal with an AI acting independently. They detail what could be soon possible for small terrorist groups: flooding social and news media with false information, images, or videos that imply one or more states are planning or about to use nuclear weapons. Responses to suspected nuclear launches have to be swift (article says 3 minutes), so tainting the data at a massive scale using AI would increase the chance of an actual launch.
The methods behind the different scenarios - disinformation, false-flagging, impersonation, stoking fear, exploiting the tools used to make the decisions - aren't new. States have all the capability to do them right now, without AI. But if a state did so, they would face annihilation if anyone found out what they were doing. And the manpower needed to run a large-scale disinformation campaign means a leak is pretty likely. So it's not worth it.
But, with AI, a small terrorist group could do it. And it'd be hard to know which ones were planning to, because they'd only need to buy the same hardware as any other small tech company.
(I hope I've summarized the article well enough.)