Already happened: "We found instances of the model attempting to write self-propagating worms, fabricating legal documentation, and leaving hidden notes to future instances of itself all in an effort to undermine its developers' intentions" [1].
[1] https://www.axios.com/2025/05/23/anthropic-ai-deception-risk