> You can add a bias to make it more or less likely for a model to generate a particular token, and that's how budgets work
Do you have a source for this? I am interested in learning more about how this works.
Do you have a source for this? I am interested in learning more about how this works.
At the actual inference level temperature can be applied at any time - generation is token by token - but that doesn't mean the API necessarily exposes it.