"You can force an LLM to give the same response by setting temperature = 0, which is, in general, a good practice."
I thought this wasn't true, i.e run it enough times there is a chance the output won't be the same?
I thought this wasn't true, i.e run it enough times there is a chance the output won't be the same?
Things happen in parallel and as we known not even something as basic as adding up a bunch of floats is associative. Combining that with the fact that CUDA makes few guarantees about the order your operations will be carried out (at the block level) makes true deterministic behavior unachievable.
I got ChatGPT Code Interpreter to generate an example for me:
a = 0.1
b = 0.2
c = 0.3
result1 = (a + b) + c
result2 = a + (b + c)
(result1, result2, result1 == result2)
(0.6000000000000001, 0.6, False)