Hacker News Reader

And still today we spend a great deal of effort trying to make our randomly-sampled LLM outputs reproducibly deterministic:

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

View on HN

can't you just save the seed?

My understanding is that because GPUs do operations in a highly parallelized fashion, and because float point operations aren't commutative, then once you're using GPUs the seed isn't enough, no. You'd need the seed plus the specific order in which each of intermediate steps of the calculation was finished by the various streaming multiprocessors.

Reply on news.ycombinator.com