Phidata: Build AI Assistants using function calling
github.com
github.com
Does anyone know how the LLM vendors are actually implementing function calling? Is it just a thoughtful prompt and a loop where they parse outputs and check if it corresponds to the arguments of the function? Or something else?
To answer you first question, function calling models are trained to detect when a function should to be called (depending on the input) and to respond with JSON that adheres to the function signature.
A thoughtful prompt and a loop where they parse outputs and check if it corresponds to the arguments of the function is the ReAct model of building agents, which I've found starts to deteriorate in quality after about 5 tools.
Whereas with function calling, because the LLM is trained to detect/call functions it can work very well up-to 30-40 functions -- after which we need to start using techniques like delegation.