Hacker News Reader

Comment by wolpoli | Hacker News Reader

For this specific example, an LLM in front of NICE will produce the correct result. It's a matter of time before this case will be fixed.

From a non-expert's perspective through, an LLM is very dependable, unless it completely goes off the rail. How would anyone know when to trust it and when to be skeptical?

View on HN

That's not how LLMs work. Including the NICE, if it actually isn't already, will not guaranty a "correct" result. It will increase the chance that the response is directly coming from the training but there is no guarante. If you are interested in why this is the case you can read this [1] post from Stephen Wolfram on how ChatGPT and in general LLMs work. This might give some insight on how and when to to use it more effectively.

[1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Could we instead have the LLM use NICE similar to how Bing uses the web as a reference instead? It still wouldn't guarantee a correct result, but it would that increase the reliability, right?

Could be. That means feedinng it the data from NICE in a prompt rather than relying on the data being in training set. That will intuitively increase the reliability of the answer (I have to look into it a bit more).

On one hand if you already have the NICE data at hand, you already have your answer. There won't be a need for a search enginge or a chat bot other than to perhaps, summerize the data (which is valuable on its own). On the other hand, if you don't have the NICE data at hand, the correctness of the response relies on the accuracy of the search method in order to feed the correct page to LLM. This is an issue additional to LLMs accuracy.

At first it might seem like an easy problem to solve but when one wants to engineer a solution to a nice streamlined product, it's more challenging; unsurprisingly.

Trust but verify. We’ll get processes built around this for accuracy sensitive applications. I imagine it will look something like a GAN configuration with two or more LLMs trained to adversarially critique and fact check the outputs of the other(s). Might even mimic the relationship between the hemispheres of our brains.

Two models trained from different lineages won't hallucinate the same. When you want to check for hallucination the cost is to run the task on two models. For now. But soon it looks like LLMs will be better calibrated. It seems they are well calibrated after pre-training but become less calibrated after fine-tuning and RLHF. The last stage breaks its abilities to estimate confidnce scores correctly.

Reply on news.ycombinator.com