People keep saying that "models are just compilers, and I don't see you complsining about compilers". Which is such a bullshit argument
This is the main reason (and yes, many modern compilers and CPUs carry some non-determinism which actually quite well explained and specified).
> But then, will it not get better with time?
No, it won't. Bacuse that is that is the actual literal limitation/feature of LLMs.
> I mean arent we just at the beginning of the research here?
In general? Yes. With LLMs? We can reasonably say that they will never be deterministic.
There is a way to get a non-determenistic output: first question on a temp 0 local model on a completely new session will give you the same answer. The second answer in that same session will already be different on every session (even if it's the same question).
Determinism isn't the issue; it's that the prompt does not contain enough information to know the correct way to do the thing, even if it did the thing the same way with the same inputs every time.
At what point do we normalise the message "This is a stupid line of reasoning and you should feel stupid for suggesting it, stupid!"
I mean, all the reasoned and logical arguments in the world doesn't change a religious follower's faith, but emotive ones regularly work! At what point can we start using shaming language on people who apparently don't know how neither an LLM works nor how a compiler works, but still trot out this argument as a cognitive kill switch?