Comment by wokwokwok | Hacker News Reader

Comment by wokwokwok | Hacker News Reader

Parent Full thread

It’s fun to jump on the hater bandwagon, but did you try using it?

If you look at the documentation (1), the API surface is relatively trivial and obvious to me.

Every interaction is a prompt template + an LLM + an output parser.

What’s so hard to understand about this?

Is writing an output parser that extends “BaseOutputParser” really that bad?

The parser and LLM are linked using:

“chatPrompt.pipe(model).pipe(parser);”

How… verbose. Complicated.

People who like to have a go at langchain seem to argue that this is “so trivial” you could just do it yourself… but also not flexible enough, so you should do it yourself.

Don’t get me wrong, I think they’ve done some weird shit (LCEL), but the streaming and batching isn’t that weird.

You see no benefit in using it?

Ok.

…but come on, it’s not that stupid; I would prefer it was broken into smaller discrete packages (large monolithic libraries like this often end up with lots of bloated half baked stuff in them), and I’d rather it focused on local models, not chatgpt…

…but come on. It’s not that bad.

No benefit?

You’ve implemented streaming and batched actions yourself have you?

The API is complicated.

The documentation kind of sucks.

…but the fundamentals are fine, imo.

It irritates me to see people shitting on this project when they haven’t tried it; I don’t even particularly like it… but if you haven’t actually used it, ffs, don’t be a dick about it.

If you have used it, maybe a more nuanced take than “it does not make any sense to me” is more helpful to people considering if they want to use it, or parts of it, or what the cost of implementing those parts themselves might be.

I personally think these templates (like https://github.com/langchain-ai/langchain/blob/master/templa...) don’t offer any meaningful value, lack documentation and context and fail to explain the concepts they’re using… but they at least demonstrate how to do various tasks.

It probably a valuable reference resource, but not a starting point for people.

[1] - https://js.langchain.com/docs/get_started/quickstart

> People who like to have a go at langchain seem to argue that this is “so trivial” you could just do it yourself… but also not flexible enough, so you should do it yourself.

At least for now and for the most popular usecases, this _is_ true. The framework seems as though it was written by people who had not actually done ML work prior to GPT4's announcement. Regardless if that's true or not; the whole point of a highly robust large language model is to be so robust that _every_ problem you have is easily defined as a formatted string.

The whole idea of deep learning is you don't need rules engines and coded abstractions, just English or whatever other modality people are comfortable communicating with. This is not necessarily true for all such cases at the moment. RAG needs to do a semantic search before formatting the string, for instance. But as we go forward and models get even more robust and advanced, the need for any abstraction other than plain language goes to zero.

> the whole point of a highly robust large language model is to be so robust that _every_ problem you have is easily defined as a formatted string

Using language models is about automation, parsing, etc. like any NLP task.

What you’re talking about (it would be nice) is sufficiently distant to what we have right now as to be totally irrelevant.

I agree langchain is a naive implementation, but NLP libraries are complicated.

They have always been complicated.

Not being complicated is not the goal of these libraries; it’s getting the job done.

> What you’re talking about (it would be nice) is sufficiently distant to what we have right now as to be totally irrelevant.

I disagree. :shrug: Guess we'll see who is right in like 10-20 years. It also sounds as though we're talking about different things maybe? Because a lot of automation, parsing and NLP are very much "solved" tasks for GPT-4 ignoring (important) edge cases and respecting the relative lack of progress in the domain until GPT-3.

If you need agents and stuff, then yeah we haven't got that figured out. But neither will you (general you) with your hodge podge of if statements wrapping an LLM.

If we can only see in 10-20 years, you are wrong today. Which is what the point was.

I'm not sure about Python, but in TypeScript streaming is just a for loop. Similarly, if you're using OpenAI, actions are already built into their client library and are trained against in the models, so there's nothing special you need to do. Memory is just an array!

They have some neat extras like sample selectors that can be useful — although even then, if you have so many examples you need a sample selector, finetuning gpt-3.5 is often better than using a sample selector with gpt-4 (and is considerably cheaper) in my experience.

> You’ve implemented streaming and batched actions yourself have you?

Streaming and batching really aren't that onerous to build yourself. Especially if your design goal isn't to support every single LLM provider and related endpoints. And it's the kind of boilerplate that you build once and usually never touch again, so the front-loaded effort amortizes well over time.

With that said, I do think some of the langchain hate is definitely overstated. There's pieces of it that can be useful in isolation, like the document loaders, if you're trying to spin up a prototype quickly to test some ideas. But the pitch they make is that its the fastest/easiest way to build LLM-based application end to end. That pitch I find to be dubious, and thats being charitable.

I tried using their JS package but it changes every couple of weeks in very breaking ways... on top of that it's a huge package that took up most of my CF Worker's space allowance

Fundamentals are outdated - it's all based on old GPT-2/3 models which needed a lot of hand-holding and the whole point of chains was that those models were too dumb to run multi-task prompts, not to mention that by default some tasks are executed sequentially while they can be run in parallel, slowing everything down (see how they did NER).

> it's all based on old GPT-2/3 models

Are you sure?

There are examples of using mistral eg. https://github.com/langchain-ai/langchain/blob/master/templa...

This is exactly what I’m talking about. How can you say that when there is evidence that blatently contradicts it?

This reeks of “…or so I’ve heard, but I never actually looked into it myself…”

They keep adding new models but it's a bolt-on on an underlying architecture based on old assumptions that no longer hold for LLMs with emergent abilities like GPT-3.5/4.

> The API is complicated. The documentation kind of sucks. …but the fundamentals are fine, imo.

That's damning by faint praise.

All I ask is people have a considered opinion, not a mob mentality about it.

FWIW, I implemented my own library with the features I wanted from langchain; it took about a week.

I don’t recommend people do that themselves though, unless (until) they have a clear idea what they’re trying to accomplish.

Langchain is fine to get started with and play around with imo.

Reply on news.ycombinator.com