Comment by crazygringo | Hacker News Reader

Comment by crazygringo | Hacker News Reader

Can it though?

LLM's are currently trained on actual language patterns, and pick up facts that are repeated consistently, not one-off things -- and within all sorts of different contexts.

Adding a bunch of unnatural "From Wikidata, <noun> <verb> <noun>" sentences to the training data, severed from any kind of context, seems like it would run the risk of:

- Not increasing factual accuracy because there isn't enough repetition of them

- Not increasing factual accuracy because these facts aren't being repeated consistently across other contexts, so they result in a walled-off part of the model that doesn't affect normal writing

- And if they are massively repeated, all sorts of problems with overtraining and learning exact sentences rather than the conceptual content

- Either way, introducing linguistic confusion to the LLM, thinking that making long lists of "From Wikidata, ..." is a normal way of talking

If this is a technique that actually works, I'll believe it when I see it.

(Not to mention the fact that I don't think most of the stuff people are asking LLM's for isn't stuff represented in Wikidata. Wikidata-type facts are already pretty decently handled by regular Google.)

Well that's not actually how it works - they are just getting a model (WikiSP & EntityLinker) to write a query that responds with the fact from Wikidata. Did you read the post or just the headline?

Besides, let's not forget that humans are also trained on language data, and although humans can also be wrong, if a human memorised all of Wikidata (by reading sentences/facts in 'training data') it would be pretty good in a pub-quiz.

Also, we obviously can't see anything inside how OpenAI train GPT, but I wouldn't be surprised if sources with a higher authority (e.g. wikidata) can be given a higher weight in the training data, and also if sources such as wikidata could be used with reinforcement learning to ensure that answers within the dataset are 'correctly' answered without hallucination.

In this context, these are more expert systems vs LLMs, and as you enumerate, they can be built well if built well. For example, Google surfaces search engine results directly. This is similar, but more powerful, because Wikimedia Foundation can actually improve results, gaps, overall performance while Google DGAF.

I would expect as the tide rises with regards to this tech, self hosting of training and providing services to prompts becomes easier. For Wikimedia, it'll just be another cluster and data pipeline system(s) at their datacenter.

Ah, I did misunderstand how it worked, thanks -- I was looking at the flow chart and just focusing on the part that said "From Wikidata, the filming location of 'A Bronx Tale' includes New Jersey and New York" that had an arrow feeding it into GTP-3...

I'm not really sure how useful something this simple is, then. If it's not actually improving the factual accuracy in the training of the model itself, it's really just a hack that makes the whole system even harder to reason about.

The objectively true data part?

Also there's Retrieval Augmented Generation (RAG) https://www.promptingguide.ai/techniques/rag :

> For more complex and knowledge-intensive tasks, it's possible to build a language model-based system that accesses external knowledge sources to complete tasks. This enables more factual consistency, improves reliability of the generated responses, and helps to mitigate the problem of "hallucination".

> Meta AI researchers introduced a method called Retrieval Augmented Generation (RAG) to address such knowledge-intensive tasks. RAG combines an information retrieval component with a text generator model. RAG can be fine-tuned and its internal knowledge can be modified in an efficient manner and without needing retraining of the entire model.

> RAG takes an input and retrieves a set of relevant/supporting documents given a source (e.g., Wikipedia). The documents are concatenated as context with the original input prompt and fed to the text generator which produces the final output. This makes RAG adaptive for situations where facts could evolve over time. This is very useful as LLMs's parametric knowledge is static.

> RAG allows language models to bypass retraining, enabling access to the latest information for generating reliable outputs via retrieval-based generation.

> Lewis et al., (2021) proposed a general-purpose fine-tuning recipe for RAG. A pre-trained seq2seq model is used as the parametric memory and a dense vector index of Wikipedia is used as non-parametric memory (accessed using a neural pre-trained retriever). [...]

> RAG performs strong on several benchmarks such as Natural Questions, WebQuestions, and CuratedTrec. RAG generates responses that are more factual, specific, and diverse when tested on MS-MARCO and Jeopardy questions. RAG also improves results on FEVER fact verification.

> This shows the potential of RAG as a viable option for enhancing outputs of language models in knowledge-intensive tasks.

So, with various methods, I think having ground facts in the process somehow should improve accuracy.

Isn't repetition essentially a way of adding weight? If you could increase the inherent weight of Wikidata, wouldn't that provide the same effect?

If you want to increase the likelihood that answers will read like Wikipedia entries, sure.

Is it possible to finetune an LLM on the factual content without altering its linguistic characteristics?

With Stable Diffusion, you're able to use LoRAs to introduce specific characters, objects, concepts, etc. while maintaining the same visual qualities of the base model.

Why can't something similar be done with an LLM?

Fine tuning. You can autogenerate all kind of factual questions with one word answers based on these triplets.

> pretty decently handled by regular Google

Don't underestimate though the utility of "being just as good as regular Google" at retrieving facts. For one thing, getting such things wrong is very frequently cited by LLM detractors as a major drawback of trusting them, even a little bit. If it's possible to reduce the "accidentally tells you false information" occurrence from "so frequent that people expect it" to "a total rarity" then for one thing, it would signal to many people that maybe when looking for simple answers, using something not called Google is a better use of their time. This would be very, very important to basically every big company out there (especially MS, Google, and OpenAI). Today I know asking Siri is an idiotic way to get an answer because it's slow and barely even understands the query itself. An LLM is already great by comparison at both of those metrics, and if it's good at being accurate in response, it's very intriguing.

Reply on news.ycombinator.com