Ask HN: Is there any way to stop ChatGPT from lying? | Hacker News Reader

Ask HN: Is there any way to stop ChatGPT from lying?

6 pointsreplwoacause5 comments

I spent hours feeding ChatGPT documentation on a programming language framework and providing it with detailed instructions for answering questions. I specifically told it to only provide an answer if it had learned the information during our conversation, and to reply "I haven't learned about that from you yet" if it was not familiar with the topic. Despite these instructions and numerous reminders, it continued to generate false information that had no relation to our conversation. It appeared to be inferring or extrapolating, but ended up fabricating nonsense that seemed accurate but was actually incorrect. Is it possible to prevent this behavior through prompts, or will ChatGPT always mix false information with true information?

View on HN

OpenAI have deliberately made it hard to extend or modify the underlying prompt. You can tell it not to apologize, for example, and it will come right back and say 'I apologize for my failure to follow directions.'

So while you can get it to label things and do various kinds of logical evaluation, it's almost impossible to get it to preemptively scrutinize its own output. It will scrutinize its prior output and correctly identify contradictions, mistakes, and irrational inferences...and then it will go right ahead and repeat them.

It's easy to get it to give answers containing two sentences that wholly contradict each other, and subsequently to identify the contradiction and what is illogical about it. But giving it standing orders or negative orders (like 'do not suggest $incorrect_answer') generally doesn't work, so chain-of-reasoning exercises become very frustrating.

> Is it possible to prevent this behavior through prompts, or will ChatGPT always mix false information with true information?

No, because it doesn't know what lying is. It doesn't know what the truth value of the text it produces is.

Does it have any internal sense of confidence with what it is saying though? There must be something in the model that informs it about the accuracy of its information, otherwise it would think pigs can fly.

Yes, but it's generally bullshitting. It'll assert contestable things with 90% probability, the same way it will deny it has the ability to write code in one response and then spit out a program in the language of your choice (even brainfuck).

You can persuade it of new facts, but you have to be pretty specific about it. I find it useful to work with simple 'fact worlds' from fiction.

I just found this in the FAQ too, so I guess I wasted my time.

https://help.openai.com/en/articles/6787051-does-chatgpt-rem...

Reply on news.ycombinator.com