Traditional non-chaotic systems*
LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.
LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.
The only gotcha with this is that they are theoretically deterministic, but rarely in practice.
A few examples:
- Harness specific settings that user can't control (anything from timestamp to prng seeding.
- Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice).
- Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs.
But all of these can be controlled for (at some cost) and the model can be ran deterministically.
For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.
That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.
The use of LLM in any system creates non-determinism in any practical sense.