ChapGPT Hallucinates

The Onion's recent "ChatGPT Starting To Think Journalist Could One Day Be Capable Of Independent Thought"[1] is a good reaction to these sorts of unsurprising "old news" articles repeating what other journalists have been reporting on repeatedly.

[1] https://www.theonion.com/chatgpt-starting-to-think-journalis...

I keep hearing people who are surprised and upset about this - wasn't this fact clear from the beginning?

The article makes valid points about the danger of trusting an LLM as gospel. However, I am concerned about the anthropomorphic language. The LLM is only “hallucinating” if we define “hallucinating” to mean generating text that is syntactically correct and has the correct semantic shape but the text as interpreted by the reader is taken to be factually incorrect. Fundamentally this is a reader discernment issue —- which is why, in my opinion, using anthropomorphic language to describe the interaction with LLMs or the interpretation of the generated text is problematic. The imprecise description muddies our understanding of the tool to the point that it implies agency — thereby removing agency in the form of discretion from the user.

This whole category of blog post where people ask ChatGPT about themselves and then act surprised when it gives inaccurate responses is really uncompelling. The purpose of this technology is to grasp and synthesize information on general topics of use to everyone – science, engineering, literature, language/semantic competency, etc – not to have godlike knowledge of every person who ever existed. As a feature, the only person who would expect or want ChatGPT to dispense accurate information about some random blog poster is said blog poster.

I'll concur with the piece, definitely. It seems to completely hallucinate when giving any reference link, or at least GPT-3 did. I don't know if GPT-4 continues the behavior. GPT-4 is underneath Bing Search, so if it does, that'd be bad news for MS.

I assume the title of this blog is a typo. Anyway, I'm not sure how interesting a blog post using an out-dated version of the technology, and telling us something we've known from day one is.

Updated the title. Add added the following note.

Note: There was a typo in the title - Instead of ChatGPT, it was ChapGPT . I have corrected the title, but the URL remains the same because Cool URIs don't change

Asking it to summarize fake news urls where you give it a made up title as part of the seo url is simplest way to see it hallucinate.

It’s unreliableness is exhausting since you are always doubting its output.

Water is wet.

That's what these models do. That's how they work. The fact that you can sometimes (often?) end up with factual information is a happy byproduct.

it's going to be interesting to see if with all the sudden hype and income around llm people will manage to let them acquire a sense of "facts".

From the gpt4 paper it looks like they already managed to get some interesting improvements, but it's quite likely some fundamental mechanism remains to be added..

I wonder if back in the day there were breathless articles with headlines like “software has bugs!!!”

I saw a similar report that it makes up academic papers, and so since then I have been fact checking when I use it to find research.. and every paper it has cited to me has existed.

Perhaps it is a question of how relevant the subject you are asking about is? For instance the example that made me dubious was about eating banana peels for medical purposes, quite esoteric and not able to be confirmed by any outside sources.

In this case, it seems GPT is familiar with the blog but has fuzzy details that are not corroborated by any other sources.

It is a sort of dunning Kruger effect where the AI does not know the limits of it's limits but acts confidently regardless.

The title has a typo

You reckon?