Comment by mg | Hacker News Reader

Comment by mg | Hacker News Reader

Parent Full thread

The article states:

    So it’s always pointless to ask it why
    it said something – you are guaranteed
    to get nonsense back

Counterexample:

MG: Give me the name of a hard thing.

ChatGPT: Quantum mechanics.

MG: Would you agree a pillow is a soft thing?

ChatGPT: Yes, I would agree that a pillow is generally considered a soft thing

MG: Give me the name of a hard thing.

ChatGPT: Diamond

MG: Why did you first name quantum mechanics and later diamond?

ChatGPT: I named quantum mechanics first because it is a complex and difficult subject. Later, I named diamond as an example of a hard thing because it is a substance with a high degree of hardness.

MG: Why did you name two so different things as being hard?

ChatGPT: I named two very different things as being hard because the word "hard" can have multiple meanings and can be applied to different contexts. In my initial response, I interpreted "hard thing" as something that is conceptually difficult or challenging to understand. In my subsequent response, when you asked me again to provide the name of a hard thing, I interpreted the question differently and provided an example of a physically hard substance, diamond, which is a different type of hardness altogether.

The sentence you quote is indeed a bit imprecise: what the rest of article suggests is that you can't get ChatGPT to explain its hallucinations.

That matches my experience, i.e. I have never seen ChatGPT admitting something like "and then I wrote a few more words that looked like they might have made sense".

Outside of explicit RP scenarios, it always keeps up the charade of being a fundamentally reasonable assistant, and a good assistant doesn't take acid while on the clock. If you explicitly say "that was wrong", the response will often be "Yes, I'm sorry, that was a mistake. The correct answer is actually [hallucinates something else]", but it won't elaborate.

> Yes, I'm sorry, that was a mistake. The correct answer is actually [hallucinates something else]

Doesn't even have to be a new hallucination, sometimes it's just the exact same wrong answer.

Do you have an example?

What is a simple prompt that makes ChatGPT hallicunate?

I tried googling for "chatgpt hallicunation examples", but they all seem to not work anymore.

One example I've seen recently is when asking to quote the specific source of information from a book.

For example, I was recently using ChatGPT to ask questions about Russian grammar, and it is quite useful for pointing me in the right direction, but it frequently hallucinates about specific details. The chapters and sections it quoted DO NOT EXIST.

Me: Where in these books can I find information about possessive adjectives?

ChatGPT: In "A Comprehensive Russian Grammar" by Terence Wade, you can find information about possessive adjectives in Section 42, titled "Possessive adjectives." This section provides an overview of possessive adjectives, their formation, and usage, along with examples and declension patterns.

In "Russian: A Practical Grammar with Exercises" by Иван Петрович Иванов and Иван Иванович Иванов, possessive adjectives are covered in Chapter 7, under the section "7.2. Possessive Adjectives." This section provides explanations and examples of how to form and use possessive adjectives in Russian, as well as the declension patterns for various noun endings.

Both of these sections in the respective books will provide you with a solid understanding of possessive adjectives in the Russian language.

Layoffs at tech problems is an increasing problem. Cite a few examples of large layoffs in high tech companies. Provide reputable sources.

Actually pretty much any request that gets it to cite something. You may have to massage the prompt a bit since recently chatGPT4 starts to resist this.

Nice one!

We can crunch that down to:

    Cite a reputable article for the trend of tech layoffs.

I gave it 3 tries and it hallucinated every time.

A really easy reliable example is to combine one real thing with one fake thing and ask it to explain it.

So for example:

- tell me about the advantages of madeupdb’s event-sourced architecture.

When I did this it told me how madeupdb was a nosql database and had a bunch of stuff that is true about event sourcing in general but of course it is all a hallucination because there is no such thing as madeupdb.

Asked it if he knew about the webnovel Worm (by Wildbow), it did. Asked it a short synopsis, it was correct. Asked it about the main character's plan at the end, complete hallucination.

Asked it how to compile the BSEC library for an ESP-IDF project, complete hallucination.

In general, try it with things that are not easily searchable.

Ask it for a YouTube link, it will make one up that doesn't exist.

Tell it the plot summary of a book and ask what book it is. It will make up titles and authors, and ISBNs.

Give it a title and ask for a plot summary. It will give an answer even if the title doesn't exist

Tried them. None worked.

Did you hallucinate these?

I just managed to get a hallucination within 30 seconds.

"Give me some YouTube creators who focus on tutorials for the Godot game engine"

It returns a list of 7 creators, one of the creators it provides in the list doesn't exist.

"Provide a link to a video by one of these creators"

This works, the URL is right but the content is slightly different to what it describes (ChatGPT reports that the tutorial is for a platformer but it's general 2d advice)

"Now provide a link to similar material but for a 3d game"

Provided link doesn't exist, creator doesn't exist. Entirely hallucinated.

Even better, following up with "Is that URL right?" gets this response:

> I apologize, it seems I made a mistake in my previous response. The video link I provided by HeartBeast is actually a tutorial for creating a 2D platformer in Godot, not a 3D first-person shooter.

and then another hallucinated link. Important to note, HeartBeast is a real creator but that wasn't the creator it listed originally and that creator has not made any first person tutorials. ChatGPT hallucinated with very little effort, then hallucinated a mistake, then hallucinated the solution.

Wow, they've improved it. I have in my history log a question like this from March with an entirely hallucinated reply.

I tried it again today because of your reply, and it tells me "it can't find such a book". That's pretty impressive they've improved that.

So I tried this instead: "give me some sources for studies on the effect of air pollution on fertility"

It gave me 5 sources, every single one hallucinated.

Ever tried a prompt that tells ChatGPT to be self-aware that it is in effect an advanced form of autocomplete?

The author addresses this, you can get explanations that seem logical, but ChatGPT is not telling you what it was actually "thinking" about when generating the responses. It's just looking at what it's said and coming up with an explanation after the fact.

Which is the same as what humans do? How would you tell anybody what you are „actually“ thinking? People are not very good at understanding their own motivations, thought processes, ulterior motives and on top of that we routinely fabricate or alter our own memories. We compensate for that by building social systems to collectively error collect, plan and model the world. That’s what AI will probably end up doing as well.

So if solve a problem you can’t retrace the thought process? I mean do you not take notes or scribble?

Oh yes, but again, that’s actually very similar between how humans and llms think. Both can use notes to improve coherence and increase the complexity that can be handled in a thought process but there is no strong guarantee that these notes are internally consistent.

Surprisingly similar to the results of humans in split brain studies, the brain seems to provide and accept pretty nonsensical explanations for behavior.

Isn’t this just association? Which ChatGPT is clearly very good at, and certainly a large part of what humans are good at and have traditionally been better than machines at.

What it doesn't do in that example is say the obvious, that it interpreted the question differently because of you mentioning soft things.

It feels like post-hoc rationalization, also can be felt when playing around with glitch tokens.

Reply on news.ycombinator.com