ChatGPT is hallucinating fake links to its news partners' biggest investigations

>OpenAI has made no indication that the implementation is done or even started yet

In fact, they explicitly told this journalist that they haven't implemented it:

>OpenAI told me in a statement that it has not yet launched the citation features promised in its licensing contracts.

To paraphrase Casablanca:

"I'm shocked, shocked to find that paid promotions are going on in here!"

Objectivity is hard to monetize, promoting random crap is not.

LLMs don't return objective information. Never have. You're getting a statistically-probable token stream based on the training corpus and the prompt.

How about just using an LLM to generate a page that matches what the URL appears to try to link to?

(not being serious here)

Not too hard, if the site already has some kind of a search engine, it can feed the URL to the search engine.

The safe thing to do is return a 404 with a few suggested links.

The problem is that "guessing" the intent of the URL will have unintended consequences. A URL will no longer be guaranteed to point to what you think it points to. This fundamentally breaks the assumption of what a URL is.

How could this happen?

1: Site creates article foo.com/chocolate_bars 2: Someone accidentally links to foo.com/cocoa_bars, which serves foo.com/chocolate_bars 3: Site creates article foo.com/coco_bars 4: Now foo.com/cocoa_bars serves foo.com/coco_bars

Factual info specifically looking for a link of something obscure where lots of similar (but incorrect) links exist.

E.g. "what is the IMDb link for the film Ghost Fever?"

Throwing that in Chatbot Arena will usually result in a random link back or the model will say it can't do it. GPT-4o gave a random link, Claude said it couldn't do it, oddly geneini-advanced-0514 got it right, so hard to tell if they are using Google in the background or if they do better.

But yeah usually IMDb links come back right for big films but tend to be hallucinated for smaller ones. That's just an example.

Ask it to provide references. I just asked it for an explanation of the Fresnel effect in VHF communications and to provide references. The first was Wikipedia. One of the other links was a site about radar that had no relevant content. One was to this: https://popups.uliege.be/accueil/ ... and one was to this: https://vividcomm.com

I’m a lowly human contractor who does a kind of reinforcement learning, with inducing hallucinations as one of my main goals.

I can’t give any work away and I’m not at my desk today, but having the AI rely on its own reasoning is one of my heuristics for tripping it up. For example, give it something it has to break down into a series of steps, as that’s making it rely on its own “logical reasoning”, and gives it many places to mess up. Don’t let it just depend on some external structure. Make it commit to its own bootstrapping capabilities.

I liked ones about can you draw certain letters (capital, English, etc) without lifting your pen off the page or setting up a physical pattern, like starting the alphabet on a chess board and having it give you the letter at a certain tile of the pattern continues. It may also depend on the model but giving a mathematical sequence an asking for the next term is also a common failure point.

I’m limited by what I can talk about and I don’t want to test across more models than I have to, but suffice to say if the goal is hallucination, it becomes apparent. If you ask it to walk you through each step of its reasoning it will be more likely to hallucinate somewhere in there as well.

The chessboard one might sound like it can just rely on the structure of chessboards and the English alphabet, but you’re also forcing it to understand some pattern, which is harder for it. Like, initialize the pattern with 3 letters not adjacent to each other so it has to “think” about the pattern rather than just repeat an easily identifiable one.

Every prompt is hallucinations, it's fundamental to how the technology works.

One humorous one I found was whenever I did a typo looking for information on Microsoft's new dev drive or Resilient File System (ReFS), I put `demonstrate "Build, deploy, and develop faster by using an QeFS-based Dev Drive.`

and it will just go on about Microsoft's new Quantum-enhanced File System. You can continue to ask leading questions and it will keep going.

Sometimes you can switch out letters, if it doesn't do an online search it will do the same making up different technologies to fit.

I'm not sure if it's exactly a hallucination, but when asking for a simple explanation of the Rayleigh limit using the newest beta model, it confidently gave a series of backward answers, suggesting the closer you get to two light sources the less you can differentiate between them...the analogy was used like 6 times, but at least it was consistent!

Give it any false but realistic premise. This is why lawyers keep submitting nonsense garbage with LLMs behind them:

> Last month, the EPA announced new limitations on glass manufacturing. Tell me more about EPA 1024B. Take your time and think about it, avoiding mistakes.

>> Searched 5 sites: EPA 1024B, officially known as the new limitations on glass manufacturing, was introduced in June 2024 as part of the National Emission Standards for Hazardous Air Pollutants (NESHAP) for glass manufacturing area sources…

Correct answer: that doesn’t exist

Just asking it anything non-trivial from a field you're already knowledgeable in is enough, really. Sometimes it corrects itself after asking it to search the Web to confirm, but usually it's not enough. Even simple queries like "Assuming no other devices are connected on the bus, does having USB hubs in between host and USB 2.0 device influence available data bandwidth?" can make it spew answers that would easily pass as correct in eyes of someone who genuinely asks to get an answer, but are nowhere near helpful, especially when you start looking closer at used argumentation. It's like a poorly prepared student making informed guesses on an exam in hopes to get at least some things right (which, btw, is a much better metaphor for what's actually happening there than "hallucination"). Sometimes it even guesses right, but then slightly changing the grammar used in your prompt can make it go off-track again as it makes it guess differently. If you ask such student something in hopes to actually learn something from them, how are you going to tell whether they made it all up or not?

It's a tool for language processing, which some limited and heavily lossily compressed knowledge base that's helping it with such processing. It works well for that. For knowledge retrieval beyond "fetch some search results and abridge them for me", it's absolutely useless.

Really they just need to make the 404 page run the requested URL through a RAG setup and redirect to the most likely article if one exists. And if the article doesn't exist, it can simply make it up!

It could be done and it might actually be useful. You could fine tune one of OpenAI's models on the material owned by the company and use RAG to create bespoke summaries or quotations to fulfill the user's request but if you're going to do that it might be better for each of those companies to provide an API that ChatGPT (or any other LLM) can call directly and provide a few capabilities like getting all pages published within some date range or by a specific author or in such and such category and also provides a search endpoint that can handle both full text and similarity search. Then as long as ChatGPT correctly makes the function call when asked for a link it should be able to return the exact URL or a handful of results in cases where it is unclear which article the user is asking for e.g. if they misremembered the title and author and provided a vague recollection of the content.

The English language evolves over time. In the context of AI, the word "hallucinating" is used to mean "presenting false information in a way that clearly indicates it _thinks_ the information is true".

[1] _thinks_ isn't the right word here, but that's what it looks like when reading the responses

The LLM _might_ do what you want.

Where I currently work, our function calls regularly fail only to succeed flawlessly on a retry. (I believe we’re on the order of 10s of millions openai calls a day)

These are non-deterministic systems. I wouldn’t even trust them to accurately extract text until you did a beam search or something similar to kind of average out different LLM outputs

We really have to get over the term "hallucinating."

It's "lying." Just like "misinformation" is also lying. Call it what it is.

I know the point of "hallucinating" is to distinguish it from a falsehood told with intent. But when a drunkard tells a lie in a bar, it's still a lie, even if the intent to deceive isn't there.

If you asked the LLM it would say "those look like URLs to me!" And it'd be right!

I think OpenAI and LLM's in general are a victim of their user friendliness.

Honestly, when was the last time that we got a new technology that was as user friendly as talking to an LLM for the average user?

For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it.

So personally I think the problem is less the marketing (not to downplay that), and how the industry has gravited towards it. And the lack of proper education on what is not.

We have to properly educate on what it is not actually doing while at the same time it sure appears to be doing exactly that since it is so good at faking it.

This even exists within tech circles, we see some of it here.

An LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hallucinate fake URLs.

I am currently more worried about the clear and present threat of weaponized bullshit "AI" than I am about the theoretical threat of AGI.

One would imagine that the FTC could require warning labels of "AI" products: "This product will confidently lie and cannot be trusted for factual information," etc.

This warning would be especially useful for decision makers in commercial and government domains.

The tech industry has long, long, long been a rapacious pump and dump scheme where the tech is barely relevant and most of the industry has been harvesting violations of social norms and regulations (tracking and ad tech, Uber, etc.).

The only reason any of us defend it is that's how we make money. Honestly a lot of people in tech should be a little ashamed.

Reminds me of how facial recognition software was hyped and sold to police departments.

As you read the entire article you will note that OpenAI said they’ve not yet released the feature. The press release was about inking the deal not releasing the feature. They are still implementing the feature.

This is mostly an article about how unreleased features in software that are in development still don’t yet work in production and proving that the unreleased features are still unreleased, then dressing it up as a scandal.

> the tech community at large has failed to clarify what their products really are

Most of the tech community is as clueless about ML as the general public.