LightRAG: The PyTorch Library for Large Language Model Applications | Hacker News Reader

LightRAG: The PyTorch Library for Large Language Model Applications

github.com

82 pointsbbzjk714 comments

They define RAG as Retriever-Agent-Generator in contrast to the rest of the industry, which defines it as retrieval-augmented generation. So are they creating agents, or doing good old RAG?

Scanned through the docs quickly because I had the same question. Funny enough they are trying to re-define RAG to included Agents which is a bit frustrating since it’s a pretty well known acronym at this point.

Even funnier, they somewhat have real RAG in there under the label of retrieval (thoiugh that is more fair since the opposite is also true, retrieval and ranking for search and knowledge type tasks had names before and are frequently clumped under RAG nowadays).

As an aside, the docs are a bit hard to read since they look translated form a different language or just need a pass from someone who speaks English well.

So it’s not RAG as is currently popular -but an ML approach focused lightweight LLM framework that attempts to circumvent the over engineering of LangChain et al?

I think it’s an attempt merge traditional RAG with PyTorch style

The authors linked in post seems to give some more details on the intention of it:

“The problem with researchers’ code is that it has no abstraction at all. It only works on the benchmarking dataset. They lack powerful string processing that can generalize the tool calls. Their code is hard to read as it is not organized. Thus, it is really hard to adapt to production.

The problems with the frameworks are that they have abstracted so many layers but are not supported with really strong engineering to provide robust adaptation to different use cases. They are more like no-code solutions, which are perfect for people to use out-of-the-box, but terrible for products and research.

There can be a solution great for both the ML research and the product community: a library that provides strong string processing, an easy-to-understand tool interface to support tool calls, various output formats, and various access to different models with great monitoring capabilities like TensorBoard for Pytorch. That is the mission and purpose of LightRAG. Let researchers and engineers work on what matters: prompts, datasets, evaluations, and model finetuning without building everything from scratch alone with subpar solutions or using an overly engineered framework which requires them to spend more time figuring out how to customize instead of on what matters.

I know I'm not a popular voice, but this is super important, and I think the majority of the community is moving in the wrong direction, even some of those people I looked up to and considered the best in the field. If you are passionate about a library that really unites RAG and Agent research and engineering, please work with me. I know I'm not able to do this alone”.

https://www.linkedin.com/posts/li-yin-ai_both-ai-research-an...

It feels the title should be LightLLM or something, was expecting document ingestion etc in the docs but were missing.

That would also be somewhat unfortunate naming, considering the existence of https://www.litellm.ai/

If you're okay with repurposing RAG, you're not going to cry over LiteLLM :)

I’m pretty sure PyTorch is the PyTorch for LLM apps and I wish people would stop spamming HN with their do-nothing AI hype repos.

PyTorch is still a bit too low level for LLM applications compared to something like Hugging Face transformers which abstracts best practices and does a good/performant job of it.

I don't see what this has to do with PyTorch.

...or RAG!

This is kind of cool! Seems more lightweight than alternatives, with less moving parts.

Also, some comments are complaining about the RAG as "Retriever-Agent-Generator", which is not the accepted definition, I think this must be a translation error. In another page[0], the correct RAG definition is present.

[0] https://lightrag.sylph.ai/tutorials/eval_a_rag.html

Folks, let's focus on the value instead of just bashing their name and comparing it to PyTorch. The team has an AI background, so naturally, PyTorch feels like home to them. Given the state of the existing libraries, it is great to see another one taking a completely light approach.

Neat, definitely aligned with this philosophy. If you like this, you'd appreciate Substrate - an even lighter weight, Pulumi-inspired API for modeling these workflows: https://x.com/vprtwn/status/1810363923750646032?s=46&t=HBHv3...

No openai required ?

Reply on news.ycombinator.com