This is great! Does anyone know if the llama models are trained to do function calling like openAI models are? And/or are there any function calling training datasets?
In general, you'll do a ton of damage by constraining token generation to valid JSON - I've seen models as small as 800M handle JSON with that. It's ~impossible to train constraining into it with remotely the same reliability -- you have to erase a ton of conversational training that makes it say ex. "Sure! Here's the JSON you requested:"
What are you doing underneath, here? If thats secret sauce, I'm curious what you're seeing in tokens/sec on ex. a phone vs. MacBook M-series.
Or are you deploying on servers?
respond in JSON in the following format: {"spam_score": X, "summary": "..."}
and _then_ you constrain the output to json, the quality of the output isn't affected.
https://www.llama.com/docs/model-cards-and-prompt-formats/ll...
> With text-only inputs, the Llama 3.2 Vision Models can do tool-calling exactly like their Llama 3.1 Text Model counterparts. You can use either the system or user prompts to provide the function definitions.
> Currently the vision models don’t support tool-calling with text+image inputs.
They support it, but not when an image is submitted in the prompt. I'd be curious to see what the model does. Meta typically sets conservative expectations around this type of behavior (e.g., they say that the 3.1 8b model won't do multiple tool calls, but in my experience it does so just fine).