> we know everything about how LLMs work
No we don't. That we understand the low level mechanics of a system doesn't mean we understand how any high level phenomena emerge from those low level mechanics.
This is as true for quantum mechanics as it is for LLMs.