For posterity, this is the "JSON mode" mentioned at the bottom of the post.
The docs say it's on by default if you use function calling normally: https://platform.openai.com/docs/guides/text-generation/json...
> Note that JSON mode is always enabled when the model is generating arguments as part of function calling.