HuggingChat – ChatGPT alternative with open source models

It's been pushed yesterday. Works just fine.

It's called retrieval augmented generation (RAG) and there's no extra training. The data (e.g. web search result) is given as input to the LLM.

If you search for "retrieval augmented generation" you'll find papers, tutorials, videos etc about it.

Web search is not a capability of a “bare” LLM, but in an LLM-based system it can be done by giving the LLM access to a “web search tool”, I.e essentially you instruct it to output a specific structured text (typically json but doesn’t have to be) indicating its “intent” to search, and your wrapper intercepts/detects this structured response, does the actual search and returns the results (e.g snippets from top k results) into the context of the LLM amd have it use these to respond to your question.

A similar thing can be done with external documents - your wrapper retrieves docs/fragments relevant to the query, puts them in the context and lets the LLM use them to answer the query. This is called Retrieval Augmented Generation (RAG).

The above is a highly simplified description. In the Langroid library (a multi-agent framework from ex-CMU/UW-Madison researchers) we have these and more. For example here’s a script that combines web search and RAG:

https://github.com/langroid/langroid/blob/main/examples/docq...