There are emergent behaviors and properties that make us more important than simply the sum of our chemistry - at least I hope.
But what if LLMs, could there be emergent behavior there that makes it more than simply the sum of its code?
Now I hear gravity must be emergent. What a useful tool to sound smart.
We don't. I don't think many try to cover that up.
Emergence just means that a system has properties not apparent from the parts going into it. It's not a tool to handwave away that fact and ascribe it to "emergent hocus pocus", it's a tool to frame the search for why that happens.
Ah, and there we have it.
Emergence is just complexity. The problem with complexity is it takes a lot of time and massive amounts of effort so determine what it is in the computation (digital or quantum) that produces the effect, and is commonly not reducible.
Bringing in an imaginary character in order to explain the complexity is just a failure of logic.