Hacker News Reader

Comment by osener | Hacker News Reader

Interesting project, thanks for submitting it!

Obviously not commenting on this project as I’ve yet to read up on it, but what are your proven to be useful building blocks for LLM apps? This space is super new and everyone is experimenting, promoting their solutions and trying to build communities/stargazers/customers around them. It is getting hard to distinguish what’s just fluff and what is an actual improvement over raw OpenAI APIs.

Even the more popular projects such as LangChain and TypeChat are a bit hard to rate regarding whether they’re actually quality higher level libraries worth adopting or if I should just read their source as inspiration and build my API calls myself.

I would really appreciate suggestions!

View on HN

Zep author here. LangChain is a great framework and has a very broad and active ecosystem. Many of LangChain's core components, such as chat history "memory," history summarization, entity extraction, vector search, and more, don't scale well in production. They operate in memory and often synchronously within the chat loop, resulting in poor user experiences and limiting deployment options.

You can use third-party ecosystem integrations with external SQL databases, vector databases, etc to fix some of these issues. This requires some know-how, infrastructure, and time. Zep's building blocks are turn-key solutions to these challenges and offered in a single service.

The Zep GitHub project, website, and demo video provide a good overview of the project's functionality and how the service solves these issues.

This blog post on the LangChain website offers some benchmark data using Zep vs LangChain's core memory components: https://blog.langchain.dev/zep-x-langchain-slow-chatbots/

Since you are targeting developers I would highlight the differentiating features: "database-backed", and "asynchronous", or whatever they are. This is more informative than "production-grade", "fast", or "scalable", which anyone can claim. In fact, the competitors have a better claim to being fast -- they're in memory! -- so I would drop them unless I had benchmarks. Something I would also encourage you to work on.

Finally, call out your competitors, and explain your value add for each one. Maybe in a table. Why not just use a vector database and hand roll the rest, for example?

Good luck!

Thanks for the advice! Some benchmarks here: https://blog.langchain.dev/zep-x-langchain-slow-chatbots/

That's not really benchmarks. To me, it just shows why putting things like langchain in the mix makes your life harder in the long run. Having to add more tools is not an ideal solution. What people want are actual "building blocks" that don't require adopting a framework

Great question. IMHO it's better to write something yourself. These things are not complicated and you can have better control and understanding by writing it yourself. Once the industry settles on a set of tools, you can start thinking of migrating if needed.

Zep has Python and TypeScript packages for teams building without LangChain: https://docs.getzep.com/sdk/

Reply on news.ycombinator.com