> A LLM is smart enough to [...]
I thought this was an interesting juxtaposition. I think we humans just naturally anthropomorphise everything, and even when we know not to, we do anyway.
Your analysis is correct, I think. The reason we find this behaviour frightening is because it appears to indicate some kind of malevolent intent, but there's no malevolence nor intent here, just probabilistic regurgitation of tropes.
We've distilled humanity to a grainy facsimile of its most mediocre traits, and now find ourselves alarmed and saddened by what has appeared in the mirror.