Hacker News Reader

Comment by thorum | Hacker News Reader

I wonder if the full prompt is actually this long, or if GPT-3 is just predicting the 'most likely' next 5 sentences when asked.

View on HN

Exactly this. I like testing chat gpt’s hallucination and my most recent one was to ask it to convert a conversation I had so far into html. It followed the prompt correctly but instead of inserting its actual answer that it had given into the html doc, it started giving different answers which included hallucinating new information.

Again, the problem here is there is no way to know for sure if what these ai chat tools are saying is correct or imagined.

At some point though, that's just true of the universe.

For millenia we thought planets orbited the Earth and there were complicated and sophisticated models to predict their motion (see epicycles). But we were just hallucinating.

Then Newton hallucinated his universal law of gravitation, which worked pretty well until Einstein hallucinated relativity.

That's completely different from asking GPT to perform a deterministic, lossless operation only to get output that doesn't follow from the input and instructions.

We're well into a relaxed-determinism future.

For those (like me) who are wondering:

> In artificial intelligence, a hallucination or artificial hallucination is a confident response by an artificial intelligence that does not seem to be justified by its training data when the model has a tendency of "hallucinating" deceptive data.

https://en.wikipedia.org/wiki/Hallucination_(artificial_inte...

I'm skeptical. We knew that the leaked prompts for ChatGPT were genuine because it was leaking the actual current date; putting in the current date was necessary to stop it hallucinating about future events having already happened, but provided a highly reliable marker of prompt leaking - obviously, there's no way it could repeatedly hallucinate the actual current date without that being based on its prompt to some degree, and if it copied that part right repeatedly and reliably, then the rest is probably genuine too.

But in this case, the supposed current date in the last screenshot is 30 Oct 2022, which is nowhere close to 8 Feb 2023.

Perhaps if it was retrained on a corpus that included its own prompt, it might be hallucinating that a date in 2022 is what's most likely to come after it has regurgitated its prompt!

It's possible the work started in October 2022 and only became public knowledge in 2023.

Shrug. Just sayin'.

p.s. Gwern, your website is prolific. Thank you!

No, because ostensibly that is the date for the current context, which should be sometime in February if not today.

Is it that hard to believe that Microsoft hard coded the start date rather than making it dynamic?

What would the function of including a start date be if you don't give it the actual start date? The lazy thing to do would be to omit to the start date altogether, not hard code it.

One thing that also makes me hesitant is that the model is never informed concretely how it can search the internet. Presumably, you would require a specific syntax for it to launch a web search (such as [search:{query}]), but it is never informed of that syntax, just that it can search for things.

Also: "While Sydney is helpful, it's actions are limited to the chat box." Ominous...

That last bit may be just to prevent it from hallucinating non-chatbox actions. Or in case someone tries to hook it up to a web API via its search capabilities.

I didn't know people meant chatboxes when they were talking about the AI box experiment. ;)

> "While Sydney is helpful, it's actions are limited to the chat box." Ominous...

Its not Alexa.

From the followup tweets, it seems that the output is deterministic (same rules).

If they didn’t want it to disclose its code name, why would they tell it its codename? I don’t think there’s any way that prompt is accurately reflecting what is going on under the hood.

To figure this out, do it again but with slightly different goading. If was hallucinating, the new hallucination should be slightly different to the previous one.

Sadly by the time enough people can manage to replicate this it’ll already be patched

What gives a hint is the same answer for multiple tries and prompts. He asked 5 by 5, then lines above the date, then if some string existed.

All those times Sidney have the same content.

Big, but plausible

The detail and specificity do seem a bit more human generated to me.

or if the full prompt is that long, but one word is different, or ...

Reply on news.ycombinator.com