The probability distribution over next tokens given previous tokens is deterministic. The sampling algorithm for that distribution is non-deterministic.
So the total generation of text from an LLM can be made fully deterministic. The problem for scientists is that we cant do that in the deployed systems...