The evidence is the acts the AI has already done.
Imagine you use it to inflict trauma on your cruelest political enemy. Then next year they do that to you. That is war, and we already do it. But we don’t want people / governments in charge that are going to do this.
God knows we have governments and individuals doing this historically, and this is probably the greatest source of historic instability. It might be the single best argument for open models — a unified frontier where no single exploit is going to represent capture.
This is exactly the opposite of what Dario proposes.