This sounds to me like a straw man argument. Obviously 2+2 will always give you 4, in any modern LLM, and even just in the Chrome address bar.
Can you offer a real situation where we should expect the LLM to return a deterministic answer and should rightly be concerned that we're getting a stochastic one?