Hacker News Reader

Nondeterminism provides an excuse for errors, determinism doesn't.

Nondeterminism scores worse with human raters, because it makes output sound even more robotic and less human.

Would picking deterministically help through? Then in some cases it’s always 100% wrong

Yes, it is better if for example using it via an API to classify. Deterministic behavior makes it a lot easier to debug the prompt.

Determinism only helps if you always ask the question with exactly the same words. There's no guarantee a slightly rephrased version will give the same answer, so a certain amount of unpredictability is unavoidable anyway. With a deterministic LLM you might find one phrasing that always gets it right and a dozen basically indistinguishable ones that always get it wrong.

My program always asks the same question yes.