Hacker News Reader

Comment by nielpen | Hacker News Reader

Say "I don't know."

Back when GPT-3 was all the rage on the internet, I remember Kevin Lacker's "Giving GPT-3 a Turing Test" left a real impression on me [1]. Short read, but gets to the crux of the issue. GPT-3 is a (really sophisticated) statistical model, GPT-4 will be no different. Maybe it will showcase new emergent properties of LLMs, like GPT-3's in-prompt few-shot learning did. But the fundamental constraint of a statistical model optimized to a masking task remains -- they're really bad at introspection and confidence-assessment.

[1] https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.h...

View on HN

I've been trying out the article's suggestions, and ChatGPT is able to tell me that it does not know today's date: "I'm sorry, but I am not able to access the current date because I am a large language model trained by OpenAI and do not have access to the internet. My knowledge is based on the texts that were used to train me, but I don't have the ability to browse the web or access real-time information. Is there something else I can help you with?"

That's one of the special-cased things though. ChatGPT was deliberately taught it does not know the date, along with a few other things.

I have been impressed at how quickly ChatGPT has been adjusted to say I don’t know. You could previously trick it with extremely simple logic problems like:

> Jack is taller than Jim. Jack is taller than James. Is Jim taller than James?

Even if you asked it to explain, it would make up some bs.

Now if you ask it tells you it doesn’t have enough information to answer. More tricky problems have been resolved as well.

ChatGPT is actually quite good at this and I think it's a hint at the future of alignment. Many (most?) of the responses from ChatGPT are along the lines of: I don't know, I can't know, I can't respond, etc. It's still far from perfect but I think exploring the results of human feedback for alignment is one of the reasons OpenAI decided to release ChatGPT rather than rush straight to GPT4 (larger model, more data, retrieval, etc.).

> Say "I don't know."

That's a good answer, thank you. A similar answer might be: "Admit it's wrong". Technology really has gone in a strange direction if we end up measuring the performance of software systems in terms of which personality flaws they have.

Oh 100% -- anthropomorphic bias towards AI is about to get really interesting, I'd expect. Especially if prompt engineering comes to be part of the job description for tech, which seems likely.

Mmn, no, it will admit it's wrong if you tell it so.

Humans are bad at introspection and saying “I don’t know”

ChatGPT seems to have another statistical algorithm to assess uncertainty for the original statistical algorithm, and say “I don’t know” if it reaches a threshold. GPT-4 may add another model some statistical introspection. At the end of the day I think this is how humans calculate their own uncertainty, so a model doing this way may lead to the same results

exactly this. When it doesn't know, it will ramble, answer an unasked question or boldly lie. Better to give up in such cases; we didn't need artificial ego.

Reply on news.ycombinator.com