AI fears come out of a modern enlightenment tradition that elevates disembodied minds to some kind of godlike status. It falsely equates reason and intellect with power. In reality that's never really the case, or all the rationalist, 200 IQ people would run the world and stop all the AI risks. In reality all they do is write blog posts for each other, despite the fact that many of them are probably two standard deviations smarter than all the politicians. Closely related is that XKCD crypto meme with the crowbar that everyone knows.
Any virtual AI is going to be physically subject to humans. So unless you voluntarily build the AI a gigantic killer robot, Fallout style, how smart it is won't matter. The idea that being smart isn't all that it's made out to be just never crosses the minds of AI risk people.