Langchain is the weirdest thing. It just does not make any sense to me. I see absolutely no benefit and plenty of cost associated.
Otherwise, I have to agree. Langchain to a large extent seems to base its existence on a problem that barely exists. Outside of LLMs as services, the challenging part about LLMs is figuring out how to get one up and running locally. The hard part isn't writing an application that can work with one. Maintaining "memory" of conversations is relatively trivial, and though a framework might give me a lot of stuff for free, it doesn't seem worth giving up the precision of writing code to do things in a very specific way.
Perhaps Langchain is "good" for programming noobs who might benefit from living in just the Langchain universe. The documentation provides enough baby steps that someone who has maybe a few months of experience writing Python can whip something together. However, I'm really giving it the benefit of the doubt here. I really hope noobs aren't getting into programming because they want to build "the next ChatGPT", inherit a bunch of bad ideas about what programming is from Langchain, and then enter the workforce with said ideas.
I guess it's all about whether you believe the most recent LLMs to be good enough to do their own adequate decision making inside their own hallucinations, or if you need to enforce it externally. If the latter, you use LangChain or LlamaIndex. If the former, you rely on OpenAI functions/Claude 2 iterative prompting with minimal Python glue.
LangChain and LlamaIndex also have some nice functions like document imports, RAG, re-ranking but one can simply copy the corresponding code and use it standalone without the rest of the library.
In my experience, it is actually surprisingly hard. I guess it depends on just how "human" you want it to feel. I wrote about it here: [link redacted]
I was actually surprised that LangChain doesn’t do it this way. Just an example of how we shouldn’t assume the established implementations are the best ones and one should always be skeptical and take a fresh look. I posted about this a couple days ago—
https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-...
[1] Langroid: https://github.com/langroid/langroid
If you look at the documentation (1), the API surface is relatively trivial and obvious to me.
Every interaction is a prompt template + an LLM + an output parser.
What’s so hard to understand about this?
Is writing an output parser that extends “BaseOutputParser” really that bad?
The parser and LLM are linked using:
“chatPrompt.pipe(model).pipe(parser);”
How… verbose. Complicated.
People who like to have a go at langchain seem to argue that this is “so trivial” you could just do it yourself… but also not flexible enough, so you should do it yourself.
Don’t get me wrong, I think they’ve done some weird shit (LCEL), but the streaming and batching isn’t that weird.
You see no benefit in using it?
Ok.
…but come on, it’s not that stupid; I would prefer it was broken into smaller discrete packages (large monolithic libraries like this often end up with lots of bloated half baked stuff in them), and I’d rather it focused on local models, not chatgpt…
…but come on. It’s not that bad.
No benefit?
You’ve implemented streaming and batched actions yourself have you?
The API is complicated.
The documentation kind of sucks.
…but the fundamentals are fine, imo.
It irritates me to see people shitting on this project when they haven’t tried it; I don’t even particularly like it… but if you haven’t actually used it, ffs, don’t be a dick about it.
If you have used it, maybe a more nuanced take than “it does not make any sense to me” is more helpful to people considering if they want to use it, or parts of it, or what the cost of implementing those parts themselves might be.
I personally think these templates (like https://github.com/langchain-ai/langchain/blob/master/templa...) don’t offer any meaningful value, lack documentation and context and fail to explain the concepts they’re using… but they at least demonstrate how to do various tasks.
It probably a valuable reference resource, but not a starting point for people.
That's damning by faint praise.
FWIW, I implemented my own library with the features I wanted from langchain; it took about a week.
I don’t recommend people do that themselves though, unless (until) they have a clear idea what they’re trying to accomplish.
Langchain is fine to get started with and play around with imo.
At least for now and for the most popular usecases, this _is_ true. The framework seems as though it was written by people who had not actually done ML work prior to GPT4's announcement. Regardless if that's true or not; the whole point of a highly robust large language model is to be so robust that _every_ problem you have is easily defined as a formatted string.
The whole idea of deep learning is you don't need rules engines and coded abstractions, just English or whatever other modality people are comfortable communicating with. This is not necessarily true for all such cases at the moment. RAG needs to do a semantic search before formatting the string, for instance. But as we go forward and models get even more robust and advanced, the need for any abstraction other than plain language goes to zero.
Using language models is about automation, parsing, etc. like any NLP task.
What you’re talking about (it would be nice) is sufficiently distant to what we have right now as to be totally irrelevant.
I agree langchain is a naive implementation, but NLP libraries are complicated.
They have always been complicated.
Not being complicated is not the goal of these libraries; it’s getting the job done.
I disagree. :shrug: Guess we'll see who is right in like 10-20 years. It also sounds as though we're talking about different things maybe? Because a lot of automation, parsing and NLP are very much "solved" tasks for GPT-4 ignoring (important) edge cases and respecting the relative lack of progress in the domain until GPT-3.
If you need agents and stuff, then yeah we haven't got that figured out. But neither will you (general you) with your hodge podge of if statements wrapping an LLM.
Are you sure?
There are examples of using mistral eg. https://github.com/langchain-ai/langchain/blob/master/templa...
This is exactly what I’m talking about. How can you say that when there is evidence that blatently contradicts it?
This reeks of “…or so I’ve heard, but I never actually looked into it myself…”
Streaming and batching really aren't that onerous to build yourself. Especially if your design goal isn't to support every single LLM provider and related endpoints. And it's the kind of boilerplate that you build once and usually never touch again, so the front-loaded effort amortizes well over time.
With that said, I do think some of the langchain hate is definitely overstated. There's pieces of it that can be useful in isolation, like the document loaders, if you're trying to spin up a prototype quickly to test some ideas. But the pitch they make is that its the fastest/easiest way to build LLM-based application end to end. That pitch I find to be dubious, and thats being charitable.
They have some neat extras like sample selectors that can be useful — although even then, if you have so many examples you need a sample selector, finetuning gpt-3.5 is often better than using a sample selector with gpt-4 (and is considerably cheaper) in my experience.
https://github.com/microsoft/semantic-kernel