Microsoft's AI shopping announcement contains hallucinations in the demo

LLMs thrive in applications that involve creativity and non-serious applications mostly around fantasy or creative writing. Anyone using them seriously outside of summarization for high risk use cases is going to be very disappointed.

1) Online forums (adding 'reddit' or 'hacker news' to a search query) 2) GPT4 3) Google search

Exactly. These are models that predict text sequences. These sequences often semantically express falsehoods, but the model's not "lying", it's not "hallucinating", and it's definitely not malfunctioning. It's doing exactly what it was designed to do.

There definitely are "lies" and "hallucinations" here though ... but they're coming from the hype-cycle-hucksters trying to convince us that this whole process somehow resembles "intelligence".

If so, the difficulty is not that the model has no conception of truth and falsity, it is rather to motivate the model to tell the truth. Or more precisely, to let the model be honest, to only tell things it believes to be true, things which are part of its world model.

Unfortunately, we can't just tell the model to be honest, since we can't distinguish between responses the model does or does not believe to be true. With RLHF fine-tuning, we can train the model to tend to give answers the human raters believe to be true. But we want the model to tell what it believes to be true, not what it believes that we believe is true!

For example, human raters may overwhelmingly rate response X as false, but the model, having read the entire Internet, may have come to the conclusion that X is true. So RLHF would train it to lie about X, to answer not-X instead of X.

This problem could turn out to be fatal when a model becomes significantly smarter than humans, because this means it would less often believe according to human biases and misconceptions, so it would learn to be deceptive and to tell us only what we want to believe. This could have frightening consequences if this leads it to conceal any of its possible misalignments with human values from us.

This is way too narrow. Even if it were able to determine fact from fiction, a neural network would still be able to hallucinate as long as it has no ontology: if it doesn't "know" the boundary between objects it has no way of knowing the atomicity of its facts, so it will inevitably combine even known "facts" into falsehoods.

To illustrate, the following fact-based syllogism would sound perfectly valid in the absence of a working ontology:

  A: That green flask costs $10
  B: This flask is green
  => This flask costs $10

There are so many garbage, lazily written product reviews, by websites that only exist to get people to click affiliate links. These sites only have one goal, which is to get you to click an affiliate link and make a purchase. So it is not in their best interest to say "You shouldn't buy this."

Rather, they make a list of "top X Foobars", they start with a really expensive one, then they follow with a more reasonably-priced one, and give it a very positive review. It leads to clicks and purchases.

Given this, it's not surprising to me that even the best LLMs carry pieces of this with them. Ask it to predict text describing some tech product on a sales page, and of course parts of that low-quality data will bleed through.

Human's brains use lots of heuristics - we don't "think step by step" through everything - instead we rapidly construct an answer for almost everything.

What we say is "hallucinations" for AI in humans is "misspeaking, misremembering anything, off by 1 math/counting, missidentifying someone, using the wrong variable/method when programming, etc."

   false positive (FP), Type I error
   A test result which wrongly indicates that a particular condition or attribute is present

Edit — Though I’m not sure how well that fits for a LLM (it’s more a series of false positives at each step of prediction in the sequence).

Bullshit is probably the closest, as people will bullshit for all sorts of reasons, but hallucinations is at least intent-neutral, which I think is the point.

It's say to say stop calling it X, but then what are we supposed to call them?

It feels a bit like saying “stop calling it e-mail! It’s got nothing to do with real mail!”

There is no motive for truth, just the most likely output, even if the likeliness is low.

"Confabulation refers to the production or creation of false or erroneous memories without the intent to deceive, sometimes called 'honest lying'"

"Confabulation is the creation of false memories in the absence of intentions of deception. Individuals who confabulate have no recognition that the information being relayed to others is fabricated. Confabulating individuals are not intentionally being deceptive and sincerely believe the information they are communicating to be genuine and accurate."

https://clinmedjournals.org/articles/ijnn/international-jour...

why? "bullshitting and lies" suggests that the AI is intentionally being deceptive. "hallucinations" conveys the idea that the information is incorrect, but the AI perceives it to be correct, which is more in line with what is actually happening.

In second grade, my cousin talked a lot about flax farmers in South America, after learning about them in class. Turns out the lesson was on quinoa farmers, and he forgot the original produce and “hallucinated” the statistics about flax farmers instead. Technically the term is confabulation. Was he lying? No because he wasn’t trying to tell us fake facts.

LLMs have no intention of being wrong. Their “hallucinations” or whatever are just whatever makes sense from their statistical models. They’re really just confabulations.

> they tend to make up fake information – errors called “hallucinations.”

Hallucinations are a certain kind of error. But what appears to have happened here is a _direct_ manipulation from Microsoft. Which is a risky play by them. It doesn't take much to erode trust. People tend to trust LLMs because they tend to get things right. But if people see a few things that they know is wrong, they will quickly stop trusting. If they see a few things as marketing, then they will very quickly stop trusting.

It's not a hallucination, it is a filter. Microsoft manipulated the output to prefer their own products and boy is that a risky strategy.

Additionally belief does not mean human; for example animals can have beliefs, even very rudimentary animals. I think is more of a way of self-containing the entity and treating it as a black box.

Saying "we have no idea if it's going to spit out something accurate" doesn't sell.

"oh it's hallucinating, how cute" is an easier sell.

OTOH, it reminded me very much of my own mind (reinforced by ADHD, in my case).

This suggests to me, at least, that "the problem" isn't these models, per se. It's more like: these are probably only one module / layer in a system more similar to our brains. Just as scientists have identified distinct regions (more) involved in, say, language production, or (direct) visual perception, or etc., I'd suggest we've only just built the first substantially more practical / realistic hack / simulation (much like 3D game engines almost always use hacks - e.g., not even using the simple "Newtonian optics" model fully [i.e., "ray tracing"]) of a sort of language cortex. I'd further guess that it's going to take some maturation of a number of methods, technologies, etc. to realistically add more "cortices", but, I do think it's quite likely to happen in approx. the "decades" range...

Highly highly speculative - rather naively based on the way other technologies have developed and with a little basis in work I've done more directly in neurobio etc. No deep(er) reason / analysis, but, just my current very tentative hypothesis.

https://www.technologyreview.com/2023/05/02/1072528/geoffrey...