Ask HN: Wikipedia LLM?

3 points·sourcecodeplz··3 comments

Is there any inference project that one can locally run offline and get answers from Wikipedia?

I know duckduckgo has an implementation but as a service not as a download?

If there isn't any how much would it cost say for the English version? How about maybe just a subset?

I did do some searches but couldn't find any project like this.

No but shouldn’t be too hard to do. String together something like Lang chain with an instance of chroma running in a docker container. Just need a rest api for that, and then fork over a ton of money for embeddings and you’re done! Jk, but there is an openAI cookbook entry on doing this exact thing. Not sure about cost estimates but it will be quite expensive.

You can use the Wikipedia embedding's released by Cohere to build something pretty easily : https://huggingface.co/Cohere.

If you want a completely offline version you'd be running one of the open source LLMs locally. Otherwise put the embeddings in a VectorDB, query it for the context and send it to one of the completion APIs available (OpenAI, etc)

From the limited amount I understand training an LLM solely on Wikipedia would lead to a lower quality model as there's parts of language and intelligent communication missing (but that's my intuition). Instead a LORA could be trained on Wikipedia to have it's responses more match the style of wikipedia. Or you could feed a "regular" LLM the whole of Wikipedia and it'd answer questions.

Reply on news.ycombinator.com