What's the difference between LangChain, llama indexand others like autollm? | Hacker News Reader

What's the difference between LangChain, llama indexand others like autollm?

31 pointsjohhns413 comments

Do you use any of these and if so which one is the best to use? I know there is some skepticism around using these wrappers in the first place.

View on HN

The funny thing about Langchain is (and a lot of developers will attest to this), is the more "serious" your application becomes, or your app becomes production ready. The more you start to understand what Langchain is doing, and you will start to rewrite the custom classes to your own ones, and the custom logic to do your own stuff. Until at some point you realize, that you don't need a wrapper for an API, and can do everything with about 10% of the code.

Langchain is like a backwards 101 course into APIs for LLM's, where you start out trying to learn a new framework, but through debugging sessions end up understanding whats really being done.

Langchain is a wrapper. If you adopt the wrapper, it allows you to use a bunch of (1st party) and community implementations of LLM inference seamlessly. It allows easy integration, the thinness of its wrapper and velocty means that consistency, stability and overall architectural opinion is kept to a minimum. (For better or so worse). Langchain does everything, but it doesn't do anything that well. Think early huggingface.

Llamaindex is similar but more opinionated. Some might find it better architected. Harder to get up and running and has fewer integrarions.

Txt.io is meant to be a library with a small set of high demand features. Stable out of the box. But, it is hard to get it to work for you exactly how you need it. Less customizable. Solid implementation for RAG, QA, simple chaining style approaches.

Haven't used the others. I really like langchain. It's wrappers are thin, so you customize whichever component you want to make it work for you. Its 101-> 201 step is really annoying though. Lots of conflicting documentation and need to spend a good deal of time understanding many (unnecessary to some) layers for even the most simple usecasd.

How can it be a thin wrapper and also have many unnecessary layers for even the most simple use cases?

Without specifying what you want to do, it's a bit like asking which kitchen knife you should buy.

I think they are all relatively convoluted but can help with some boilerplate, but most of the magic is the llm itself. Setting up a vector search is not that complex, but langchain has many premade connectors for a POC.

I personally haven't seen many actual production usages (or heard of) people really using the auto part (i.e. llm -> llm -> llm feedback loop). It looks cool in theory but has many issues that make it impractical and too expensive imo.

Adding txtai to the list for consideration. Couple relevant links.

1. https://neuml.hashnode.dev/custom-api-endpoints

2. https://github.com/neuml/txtai

You might want to check out the Langroid Multi-Agent LLM app framework https://GitHub.com/Langroid/Langroid

We’ve prioritized code clarity and kept abstractions to a minimum. I recently posted about a numbering trick Langroid uses that is massively cheaper and faster than LangChain for the “Relevance Extraction” phase in a RAG pipeline:

https://www.linkedin.com/posts/pchalasani_rag-llm-langchain-...

Yeah - had problems with llama-index trying to run it. Langchain has been good so far but looking forward to experimenting with auto-gen for agents particularly.

They are racing to be the one that rules it all however the documentation gets quite convoluted. Check their Twitter, they retweet a new framework or use-case or connection every second. That makes it very difficult for me to understand what are a couple best frameworks to set these things up

What were the issues you ran into with running LlamaIndex?

Langhchain is bad code quality , badly optimized , rushed , bad quality of code version of Haystack. Haystack is proper framework with proper pipelines and Q/A architecture architecture . If you had tried and moving to production you should consider haystack.

TxtAI is also a proper framework with a lot easier interface. THey all do the same job , quality differs.

> bad code quality

Got some examples for this claim?

I found all of them to be quite bloated and annoying to use directly, which is why I made my own FastAPI based one, Swiss Army Llama. I’m obviously biased, but I far prefer it:

https://github.com/Dicklesworthstone/swiss_army_llama

Love the name! To add to it, we created rocket-röschti (rocketrösti for anyone not familiar with Swiss German) for very similar reasons. Also, back when we started there was not a lot to work with.

But obviously now, it is also a good idea looking at AutoGen.

https://github.com/rocket-science-ch/rocketrosti

Add AutoGen to the list. It’s quite new, but promising.

Reply on news.ycombinator.com