Hacker News Reader

Comment by haltist | Hacker News Reader

Hallucinations are a result of how LLMs simply generate sequences of probable tokens. Depending on instruction fine tuning and how your prompt was related to the instruction tuning dataset it might be the case that the most statistically likely sequence of tokens was to generate a date like "Jan 2022" along with the preamble about training cut off.

In general, you can not infer anything about model training and date cutoffs (or other model metadata) from the output because it might just be a statistically probable hallucination based on the training dataset.

The only way to really know any model metadata like training cutoff dates is to have someone at OpenAI actually vouch for it because they're the only ones that really know what data sets were used to train the model and whatever is the latest item included in it.

View on HN

Of course, even if the prompt given to ChatGPT is "Cutoff date: 2033-01" it doesn't mean it was actually trained using knowledge up to that date. But it was indeed provided with that date as part of its prompt so that it could use that in its responses (and it does).

I am saying even in the case that the date was given unless you have direct access to the relevant data you can not conclude the date in the output was included anywhere in the input prompts (system or otherwise).

It is pretty safe to assume that it was. Especially since it is so repeatable and the same method also shows back my own custom instruction prompts.

I find the funniest aspect of hallucinations etc to be that we've designed and trained these models based off our knowledge of biological brains and learning.

We expect these models to both act like a biological brain does and yet be absolutely perfect (ie not act like a biological brain does).

Same thing for image recognition and pretty much everything else machine: "I think that kinda sorta looks like a cat" some meatbag: "ha ha dum robot that's a dog says "you too" when the server says 'have a good meal'"

But how does this explain it knowing today's current date?

It doesn't know anything. Large language models are basically Markov chains with a large context for conditional probabilities. If the output contains the current date then it is supplied out of band in some other way. It could be part of the "system prompt" which is an extra set of tokens that modifies the conditional probabilities in the output or the output is fixed up after the fact using some kind of extra parsing and filtering after sampling.

LLMs are not magic and encoding model metadata in the output is just asking for trouble. Inline model metadata should be assumed to be a statistically probable hallucination just like all output from an LLM.

Reply on news.ycombinator.com