https://sdk.vercel.ai/docs/introduction
It uses zod for types and validation, I've loved using it to make my apps swap between models easily.
Their docs (incl. getting started tutorials) are content marketing for the platform services, needlessly pushing new users into more complex and possibly unnecessary direction in the name of user acquisition.
(I have the same beef with NextJs/Vercel and MongoDB).
Some time ago I built a rather thin wrapper for LLMs (multi provider incl local, templates, tools, rag, etc...) for myself. Sensible API, small so it's easy to maintain, and as a bonus, no platform marketing shoved in my face.
I keep an eye on what the LangChain ecosystem's doing, and so far the benefits ain't really there (for me, YMMV).
Either way I guess.
I would have thought this was impossible, I contribute to llama.cpp and there's an awful lot of per-model ugliness to make things work, even just in terms of "get it the tool calls in the form it expects."
cries at the Phi-4 PR in the other window that I'm still working on, and discovering new things, 4 weeks later
That's not to say won't need to tweak things when you cut down to smaller models, there are always trade offs swapping models.
I’d guess it supports a small set of popular HTTP APIs (particularly, its very common for self-hosting LLM toolkits—not the per-model reference implementations or low-level console frontends—to present an OpenAI compatible API), so you could support a very wide range of local models, through a variety of toolkits, just by supporting the OpenAI API with configurable endpoint addresses.
Its Apache 2.0 licensed
How is this allowed/possible in npm? Don't they have mandatory namespaces?
> How is this allowed/possible in npm? Don't they have mandatory namespaces?
No, scopes (i.e., namespaces) aren’t mandatory for public packages on npm. If a name is available, it’s yours. Sometimes, some folks have published packages that are either empty or not used, so you can reach out to them to ask them to pass it on to you. At least a few times, I’ve had someone reach out to me asking for a package I had published years ago that I did nothing with and I passed it on to them.