What a malicious human assisted by AI could do is pretty crazy too. We are giving every Joe Shmo access to super intelligence.
How is an actual AI (not what we see today, but actual AI) more scary than the headless and amoral (not immoral, but amoral) organizations we have built today?
For me it’s a question of whether the technology is potentially dangerous.
If it is, then having it in more peoples hands increases the probability of harm.