But a lot of times I think if I implement the desired functionality on my own, I'll be able to debug it more easily. Also, frameworks come and go, but the simple API call remains the same. Am I being too cautious here, or is it too early to start using LLM frameworks?
(1) Things were often more complicated than they had to be. For example, creating prompt template objects rather than just using a simple format call to replace variables in a string.
(2) Too much of the underlying mechanics got hidden away. For example, creating agents hides away the prompting to get them to use functions, so it’s hard to know why things are going wrong.
(3) Different LLMs aren’t interchangeable, so adding an abstraction for the purpose of enabling plug-and-play with different providers doesn’t really help. For example, OpenAI has function calls and streaming output, while Anthropic has a 100K context window. Those differences make abstraction less compelling, because not all LLMs are interchangeable.
(4) It limited my creativity. I found I was trying to make my code conform to LangChain’s implementations.
(5) It made things heavier than they needed to be. Instead of just installing, say, the OpenAI library, I would install LangChain with all of its dependencies.
(6) It made it harder for people to see how my code was working. They had to understand how LangChain was doing what it was doing, in addition to what my code was doing.
(7) The OpenAI API (which I use primarily) is already pretty easy to work with. Other LLM-related libraries, like Chroma, or Pinecone, are also easy to work with. So why do I need LangChain?
My two cents, for what it’s worth.
If you're comfortable with doing API calls in your own code, I'd suggest trying to prototype your desired functionality without those libraries to start with.
I think that may be hyperbolic; at the very least it helps one build a mental model and understanding of how to build these kinds of apps, and it can provide some shortcuts for fast development.