I was really impressed by how easy it is to get it to properly use such a thing, or the commands of the chat platform I was using.
Had ChatGPT been an open model, like OpenAI was supposed to produce, this kind of applications would have seemed obvious and happened in the first two weeks after release.
One issue with this approach, especially in production, is latency. You’ve got to run the entire chat through one of the big models, curie or davinci, which is not only expensive at scale, but also slow.
Then again, if you just have one or two external tools, using those big models to make the decisions which (if any) tool to call is overkill anyway. So you just fine-tune a smaller model on the task. Reduces not only costs by a fact of 100 or more. But also speeds up your pipeline considerably.