It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.
I'm not sure where you got that.
`chat.with_temperature(0.2)`
https://rubyllm.com/chat/#controlling-response-behavior
`chat.with_thinking(effort: :high, budget: 8000)`
https://rubyllm.com/thinking/#controlling-extended-thinking
Max tokens is the only one of your list that require provider specific params:
https://rubyllm.com/chat/#provider-specific-parameters
I'm one guy doing it for free. Happy to see your contribution!
I will have a deep dive into which things I felt we needed to adapt per provider.
I didn't mean to imply that you have to solve all of our wants of course.
One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
There is some difference in how OpenAI and Anthropic handle 'max_tokens'. The OpenAI way raises errors in Anthropic for example.
I will look at confirming this a bit more in depth once we've taken our RubyLLM based adapter in production and see if I can make some contribution.
Thanks for all the work! It is incredibly impressive and I never meant any disrespect.
You work your arse off for free and the guy who made the disparaging comment didn't even bother to research to see if he had the details right.
Hat's off to you Carmine for all your work. Many people really do appreciate it.
I think it is fair to list limitations from using a library that provides an abstraction; it can suggest why a tool isn't right for a person's use cases.
But it also sounds like this API handles those pretty well.
RubyLLM dev literally had to take time to provide code samples and doc links.
No issue with listing legit limitations, but be a bro and fact check claims before wasting a volunteer’s time - and potentially leading other developers on a public board astray.
I think part of the confusion with that word comes from things like corporate non-disparagement clauses. In those contracts, lawyers write the terms so broad that "disparagement" means saying anything negative, regardless of malice or intent.
I checked and it turns out I remembered correctly that setting effort and some of its settings are not portable between providers.
There are some different settings that each provider uses and in order for it to be portable, you have to force some defaults on provider A when using a setting that is almost only supported in provider B.
In our implementation we decided to drop a certain setting when using OpenAI in one case and we decided we can just force some other setting when using Anthropic. But this 'solution', might not be what others expect.
When you build an open-source library you can go this opinionated route and force these settings, or you might go the config route and force people to explicitly handle per-provider differences. I will have a look at what I am able to do in terms of a contribution and then in the PR Carmine can decide what they like.