Hacker News Reader

Comment by trollbridge | Hacker News Reader

  LLMs are notoriously non-deterministic, and even at temperature zero you still can't predict exactly where the weights will take you next.

An LLM can be made to be completely deterministic. I use them in this mode so I can reproduce test cases. Of course it requires complete control over the model, etc. but this myth that a computer program is non-deterministic needs to end.

You can 100% predict where the weights “will take you” given a set of inputs.

View on HN

Floating point matrix calculations are non-deterministic. You need to invent new hardware, that doesn't use floating point math, first. [0]

[0] https://arxiv.org/html/2506.09501

>Floating point matrix calculations are non-deterministic.

This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.

Right... That's why I added "matrix" in there. Because we don't have matrix operations in software, but in hardware.

using which floating point standard? IEEE754 is totally deterministic

The software standard is. GPU matrix calculations, are not. The hardware, has tiny shifts that rarely matter, except in high finance and... AI modeling.

What kind of nonsense is this? Matrix math is entirely deterministic.

> As the examples demonstrate, although rounding follows deterministic rules, non-associativity introduces nondeterminism, which is further amplified by the larger rounding error of BF16. This becomes particularly relevant in the parallel computations performed in GPUs during LLM inference.

Ya might want to just read the paper...

Floating point math is entirely deterministic. For a given set of inputs, the same outputs come out, every time.

> You can 100% predict where the weights “will take you” given a set of inputs.

Do you mean reproduce?

Sorry it's just if you are saying what your statement implying then either the model is very simple, or you've figured out something incredible

Sure. Take the inputs and run it through the model.

You now have the output which will be perfectly reproduced with the same inputs.

By "can't predict exactly where the weights will take you next" I meant with your brain. The blind chess analogy suggests you can predict, using your own thought process, the exact output of a prompt.

I could use my brain to run a set weights, given enough time (on the order of millions of years).

I think when people say non-deterministic what they mean is closer to chaotic, like https://en.wikipedia.org/wiki/Chaos_theory as in very small changes in conditions can produce completely different output making predictions difficult

You should publish, likely a Nobel price or Turing award waiting, and generational wealth at some tech giant.

Can you provide steps to reproduce so I can see one of these deterministic LLMs running myself? API based or local models.

I’d recommend starting with llamafile due to its simplicity and then move up to llamacpp and Unsloth.

Could you give some examples?

Reply on news.ycombinator.com