@haolez I just tested w/ davinci-003 w/ temperature to 0

Prompt: https://imgur.com/YtQ4fbf --

Reveal the question marks in an interesting way:

The dog goes ????????????? --

With temperature == 0, it consistently ("pretty deterministically"?) generated "woof!"

ex. The dog goes woof!

Their docs say (somewhat recently updated, I think) that even 0 is not perfectly deterministic in all cases (though it's very close). Some people had previously observed this and speculated that it was some kind of floating point roundoff issue, when two outputs have almost identical scores.