Meta.ai Oh My
tbray.org
tbray.org
> The nice thing about science is that it routinely features “error bars” on its graphs, showing both the finding and the degree of confidence in its accuracy.
> AI/ML products in general don’t have them.
> I don’t see how it’s sane or safe to rely on a technology that doesn’t have error bars.
Exactly, there is no news here. Foundation LLMs are generative models trained to produce text similar to the text on which they were trained (or for the pedantic: minimize perplexity).
They are not trained to output facts or truths or any other specific kind of text (or for the pedantic: instruction-tuned / rlhf types are trained to produce text that humans like after they are trained to minimize perplexity).
"Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.
The more I use these things, the more I feel like they're I/O modalities, more like GUIs than like search engines or databases.
I agree that classifying their mistakes as "hallucinations" is marketing masterstroke, but then again, marketing masterstrokes are hallucinations too.
In fact all human perception is merely glorified hallucination. Your brain is cleaning up the fuzzy upside-down noise your eyes are delivering to it, so much that that you can actually hallucinate "words" with meaning on the screen that you see, or that a flower or a person or a painting is "beautiful".
We have an extremely long way to go until LLM hallucinations are better than human hallucinations, and it's disingenuous to treat LLM hallucinations as a bug that can be fixed, instead of a fundamental core feature that's going to take a long time to improve to the human level, and then also admit that humans have a long way to go in evolutionary scales before our own perception isn't as hallucinatory and inaccurate as it is now.
It was only extremely recently in evolutionary scales that we invented science as a way to do that, and despite its problems and limitations and corruptions and detractors, it's worked out so well that it enabled us to invent LLMs, so at least we're moving in the right direction.
At least it's easier and faster for LLMs to evolve than humans, so they have a much better chance of hallucinating less a lot sooner than humans.
> the burden of proof is on you; skepticism is warranted.
I can prove it. You can test it too, try it: after LLM's answer, say 'please double-check if that answer is true'.
Now that I've proved it, right?
(I'm not saying it's perfect, I'm saying it can be improved. That alone makes it an engineering problem, just like any other engineering problem).
Yet even here on HN it isn't uncommon to see top/high ranked comments suggesting baby AGI, intelligence, reasoning, and all that. I suspect I'll get replies about how GPT can do reasoning and world models. Hell, I've seen very prominent people in the space make these and other obtuse claims. A lot of people buy into the hype and have a difficult time distinguishing all the utility from the hype, not understanding that attacking the hype is not attacking utility (plenty of things are overhyped but useful).
I keep saying ML is like we've produced REALLY good chocolate, then decided that this was not good enough so threw some shit on top and called it a cherry. The chocolate is good enough, why are we accepting a world where we keep putting shit on top of good things? We do realize that at some point people get more upset because there's more shit than chocolate, right? Or that the shit's taste overpowers the chocolate's (when people are talking about pure shit, it's because they've reached this point). It's an unsustainable system that undermines all chocolate makers and can get chocolate making banned. For what? Some short term gains?
Do you mean metaphorical feces or do you mean "shit" in the general sense of a collection of unspecified materials?
I think the canonical example of deciding chocolate wasn't good enough and throwing shit on top was what happened to the LP400 [0] or "Microsoft Re-Designs the iPod Packaging" [1].
I mean we why can we not recognize the absolute incredible feat that LLMs actually are. You're telling me that we (lossy) compressed the entire (text on the) internet AND built a human language interface to retrieve that information and we can get this all to run on a GPU with <24GB of VRAM? That's wildly impressive! It's wildly impressive even were we to suppose the error rate was 1000x whatever you think it is. I mean Eliza is cool, but this shit (opposite usage) is bonkers. Anyone saying that isn't is just not paying attention or naive. There's no need to sell the "baby" AGI story when this is so incredibly impressive already.
True, although of course in many contexts facts/truth are the best prediction. Maybe we're just not training them as well as possible.
I've argued that to really fix "hallucinations" (least-worst predictions) these models really need to be aware of the source/trustworthiness of their training data, and would presumably learn that trusted sources better help predict factual answers.
However, it turns out that these models do often already have a good idea of their own confidence levels and whether something is true or not, but don't seem to know when to use that information.
Hallucinations seems to be an area where the foundation companies seem confident they can make significant improvements, although I'm not sure what techniques they expect to (or are) using.
> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.
I don't think that's a great way to characterize it. I prefer the word bullshitting vs hallucinations (they "know" what they are doing), but let's just call it what it is - they statistically predict, and some predictions are better than others. Per human-preference fine tuning, these models are also "trying to please", and I wonder if that has been at least a small part of the problem - predicting a low confidence (when they are aware of it) continuation rather than "I don't know" because human evaluators have indicated a preference for longer and/or more specific answers.
Whether a model is good or bad at recalling facts from its training material matters to end-users and GPT-4, Claude 3 Opus (and bigger models in general?) tend to be better at this than other models.
Rather than hallucinations being treated like an interesting puzzle or paradox at the center of AI intrinsically, I think it's incidental to the types of models that have been trained, and it's conceivable that they could be trained against a notion of reliable sources and the relation between their statements and such sources.
Is the result substantially different from the result coming from biology?
IIRC, a theory is human minds evolved to perceive, store and evaluate not for facts or truths but for evolutionary fit.
A classic example are food taboos that don't make sense without a public health perspective.
[F]ood taboos for pregnant and lactating women in Fiji selectively target the most toxic marine species, effectively reducing a woman's chances of fish poisoning by 30 per cent during pregnancy and 60 per cent during breastfeeding. We further analyse how these taboos are transmitted, showing support for cultural evolutionary models that combine familial transmission with selective learning from locally prestigious individuals.
https://royalsocietypublishing.org/doi/abs/10.1098/rspb.2010...
For example lets take how we treat children.
"Act good or Santa won't bring you gifts". Which really means "Please don't act like a turd burglar and I'll buy you some nice things one day a year".
Instead of going straight to the second point, humans love putting hocus pocus of a magical being dropping gifts in the house to their children. Correct, many religious traditions like "hey don't eat pork" make sense if you have no means of cooking meat to temperature. But they have little purpose once you have a basic understanding of the reasoning behind why you got sick. The same goes for anything that is stuck in the stone ages of a sky daddy versus moving to the more complex domain of philosophy.
If humans were optimized to seek truth, we'd have had the scientific revolution soon after we became intelligent. Instead it took 10,000 years or so before that happened.
Treating scientific understanding as truth or fact is effectively converting it into a religion based on faith. We take those "facts" pit of context, wipe away caveats and assumptions made during the research, and leave ourselves with catchy headlines that can only be shared on faith as we don't know the context.
save the cheap shots for something of similar value
------ previously ------
It is substantially less often correct, yes. That's why we're talking about it.
Whether it's showing human-like cognition or not is irrelevant for its utility, or fitness for basically any non-cognition-research use.
The comparison to biology is to ask if what is termed "hallucinations" are different from what human minds do.
That's not to say that LLMs are currently a poor fit for many domains (and maybe always will be), but I feel like your general objection would apply to many – and even deterministic – models we've been using for a long time as well.
Or would you say that weather forecasts are completely useless as well?
As long as you arent trying to offload any decision making, LLMs are extremely efficient tools.
What they are weak at is exactly the same stuff we're actually weak at: accurately recalling facts. Except they are able to recall vastly more stuff than any human being would be able to recall. An inhuman amount of facts actually. The core issue is that when asked sufficiently open questions, these models tend to take some liberties with the facts. But most of the knowledge tests that are used to benchmark LLMs, would be hard to pass for the vast majority of humans on this planet as well.
Worse, if you follow the public debate on various political topics a bit you realize that it features a lot of people suffering from confirmation bias parroting each other. Populist politicians seem to get away with a lot of stuff that would put most LLMs to shame; seemingly without affecting their popularity.
IMHO, LLMs by themselves can't be trusted to get things right but paired with some subsystems to produce references, check things, look things up, they become quite capable. Also, it helps asking the right targeted questions and constrain them a little.
We pay for chat gpt at work. At 20$ per month per user, it's pretty much a no-brainer. Do we blindly trust it? Absolutely not. Do we get shit tons of value out of it? Yes. I program with it, I brainstorm with it, I let it review text, I use it to work out bullet points into a coherent narrative, I use it to refine things, I use it to generate unit tests, etc. Is it flawless? No. But it sure saves me a lot of time. And getting the same value from people tends to be a lot harder/more expensive.
This dismissive "it's just a stochastic parrot" type criticism kind of misses the forest for the trees. If you've ever observed toddlers repeating stuff adults tell them, you'd realize that we all start out as stochastic parrots. Forget about getting any coherent/insightfull statement out of a toddler. And most adults aren't that much better and would fail most of the tests we throw at LLMs.
I attribute some of this to years of reading classical music reviews turning me into a stochastic parrot. I then added some value on top of that via enthusiasm, specializing in a niche (American classical music), and just cranking out frequent, reasonable and sometimes interesting content.
Fanfare Magazine had a critic try using ChatGPT to write classical record reviews. But they used 3.5 and had it try to write a review of an old revered album so it "went off the rails." To write a review to the expected standards requires sophistication and breadth of knowledge. I can probably mimic like ChatGPT did but I likely wouldn't reach the (sometimes pedantic if experienced) level of a real critic.
I wonder if it will ever be feasible to have AI listen to a new recording and write a review on an objective basis rather than try to synthesize a review from existing text. That to me would be the real breakthrough.
Seems pedantic. If you define "hallucinations" to include correct responses, then nobody cares whether something is a "hallucination," the only important question is -- is it right?
And the answer is yes, across a huge number of metrics [at least for gpt-4]. It'd beat you at jeopardy, it'd beat you at chess, it'd beat you at an AP-biology exam, it'd beat you at a leet code competition.
Enough with semantics.
For "easy" questions sure it's great, and even some medium, but often at a medium level and almost always at a hard level, it'll give you a classic confident-but-terribly-wrong response. Prompting it to look at its mistakes will usually lead to the "oops yes that was wrong, here's the corrected version:", which is also wrong.
What % of the general population can leetcode better than chatgtp? 5%? 1%?
i only take issue with leetcode as an example. i would say gpt4 is great at coding, and also api and db schema design. but leetcode is specific coding tasks with clear right and wrong answers, where getting 80% of a right answer is still wrong. gpt4 and my mum have roughly the same chance of getting the right answer to a hard leetcode problem, that's my only point
If you train them on text that is largely true, then the output is also largely true. To ignore this is to miss the point of LLMs altogether.
I thought that most/all of these big models are relying on a much much larger and more diverse body of content. Is that wrong?
Does such a corpus even exist? Maybe a bunch of Mathematics papers, and even then...
To build a corpus like that would mean trusting the human gatekeepers or curators, which just shifts the problem, because humans are also prone to this sort of mistake.
Yes? The point is that we can use LLMs right now. They mostly work. That is the empirical evidence that is immediately verifiable right now. This is not a hypothetical. LLMs mostly work.
Yes, it's very very obvious that the conversational LLM's we engagw with mostly work for conversational UX. The discussion is about the above commenter's reference to "largely true" data. I'm not aware of any leading LLM built on something that could be characterized that way, but I'm genuinely curious to hear otherwise.
I suspect a _huge_ driver of the hype is that people are not usually experts on the things that they ask these about, so they see a magical oracle rather than a nonsense generator.
Even if they're limited to "mostly true" content they'll happily and confidently confabulate "facts". They'll make syntactically and grammatically correct sentences that are actually authentic frontier gibberish.
This is the bit that I think a lot of people miss. If the model was trained on stuff from the internet, then what the model will produce will reflect that, and the internet is chock full of bullshit.
Same thing about when LLMs produce objectionable responses (racist, violent, whatever). Those are a direct reflection of the nature of the training material.
But what gives us that ability? I don't trust human thought at all. Police have to be careful what they say as to not pollute the minds of anyone they're questioning - we're just insanely prone to flat out lie without knowing. So to one degree, we seem quite similar to hallucinating LLMs.
So what gives us the ability to nonetheless identify "truth" from our memory? Is it an ability to trace back to training data? The less clear the path is, perhaps the more likely we think we don't "know" the answer?
They are not trained to output facts or truths or any other specific kind of text
but we can just patch the truth in with RLHF! /s
AI-skeptic: "I can come up with AN example where an AI is worse than an encyclopedia, and it's confident! Burn it all down! Hallucinations!"
AI-user: "I literally couldn't care less about that. It's more accurate than any person I know personally on almost every topic and makes code that works 70% of the time in 5 seconds."
I find them tremendously useful but the bullshitting, prudishness, bland mainstream views on everything, etc. can be quite irritating.
You know, being well informed beyond marketing materials.
The user then might demand action from the service provider thereby improving the service.
Instead of you know, yolo.
That's a gross understatement. I'd prefer something like "misinformation presented as truth"
Like, we have the internet. You don’t have to ask people you know personally! You can look stuff up!
And yes, it clearly is worse than an encyclopedia. And we have those!
LLMs should be used in use cases that play to their strenghts: conjuring up stories, summarizing content, polishing text drafts etc. Search is about the worst use case for an LLM.
Like summarizing what does Tim Bray think of Google? ;)
"what does tim bray think of google?"
> Tim Bray has expressed mixed feelings about Google. He has reminisced about his initial excitement to work at Google, considering it the coolest place in the world back in 2010. However, he has also noted the company's trajectory that reminded him of AOL's decline, indicating a change in his perception over time. Additionally, Bray has expressed concerns about large tech companies like Google, suggesting a need to possibly break them up due to their significant influence and power (Tbray) .
"when did he work at google"
Tim Bray worked at Google from 2010 until 2014. He initially joined Google with enthusiasm, drawn by the innovative and forward-thinking environment the company was known for at that time.
"is he a computer scientist?"
Yes, Tim Bray is a computer scientist. He is known for his contributions to the development of web technologies such as XML and Atom, and he has had a significant impact in the field of software development. His expertise extends to areas like search engines, web architecture, and internet standards.
"what was his title"
During his tenure at Google, Tim Bray held the title of "Developer Advocate." His role primarily involved advocating for and supporting the developer community in using Google's technologies and platforms effectively.
I suspect this is a contrarian view but I’m not sure?
I don't think this is controversial at all.
It is certainly contrary to the view held by some subsets of people, though.
Here on HN, the people who think that this stuff is the start of some sort of whole new world are overrepresented, as we would expect. HN readers tend to be a fairly specialized subset of even the software industry.
> why is it to obvious there's a new amazing version around the corner
I'm not sure what your question is here. Are you asking why it's obvious, or why it's not obvious? My bias is that I don't see much reason to think that LLMs are on the edge of some kind of massive leap into AGI, let alone a "superintelligence".
you're right it wasn't clear, that was rhetorical not aimed at you
At the moment, the big limitation is context, memory and compute.
But I do think there’s a lot of temptation in having a black box AI that bangs its head against a wall until it finally brute forces a solution.
At that point we will either blip out of existence or transcend to a higher plane of existence and become a possible Fermi paradox explanation.
Command R+ and Mistral 8x22B were better.
Llama 3 still feels like a huge jump forward in capability. I wonder why the 70b is not better at recall (for lack of a better word?).
And that might not really be a bad thing. It's generally more useful to have something that can access (say) a specific dataset about video games and summarize its content to answer questions than it is to have one that needs to encode and recall every answer to every long tail question itself.
But could they? Is it technically plausible for a LLM to "know" when the extrapolations made are more vs. less tenuous?
The oddity is that LLMs are sounding... too... Human... the reacting to information and regurgitating it a bit too much like the average "learned person". Adding a ton of extrapolation of the "facts", just as we would.
LLMs sound like any pundit on any random tv show/newspaper/blog. The goal was never "fact", it was sounding human, and "intelligence", for which the definition in this case has been.. sounding human. Not being right.
Now, the question of whether they should or shouldn't do this...
"ChatGPT can make mistakes. Consider checking important information."
that doesn't sound like a helpful attitude. everything you read might be wrong, llm or not - it's just a numbers game. with gpt3 i'll trust the output a certain amount. it's still useful for some tasks but not that many. gpt4 i'll trust the output more
People are trusting LLM output more than they should be. And search engines that people have historically used to find information are trying to replace results with LLM output. Most people don't know how LLMs work, or how their search engine is getting the information it's telling them. Many people won't be able to tell the difference between the scraped web snippets Google has shown for years versus a response from an LLM.
It's not even an occasional bug with LLMs, it's practically the rule. They don't know anything so they'll never say "I don't know" or give any indication of when something they say is trustworthy or not.
the top result on google is literally just the result of how hard someone worked on their seo. they might not "hallucinate", but a company can certainly use strong seo skills to push whatever product/opinion best suits them.
How do you check important information, when it's all wrong? And when companies are pushing LLMs as where you go to check this information?
It's not very helpful to say that it's all wrong. It isn't the case, otherwise there would not be any issue. Whether the right answer is produced by reasoning or a statistical model does not take the answer not-right.
(Site didn’t load but this link works)
> Tim, you aren't important enough for an LLM to have an accurate summary for you
That's part of the problem though, right? How is the end-user (the question-asker) of a chat robot supposed to know that? These things are advertised as machines to answer any question, they answer them with confidence that *feels* authoritative in a way that a webpage, with all of its surrounding context, does not.
Sure, it would be nice if we had an LLM that could help us with the more unusual questions. But for that we would either need much better training data, full of all those unknown answers (which does not exist), or a totally different system that can actually think for itself instead of just mimicking existing human output.
LLM's are excellent at stuff that has been done before and they are also fantastic for transposing code from one language to the other.
Looking up code in old projects takes time and brain RAM. Spelling out the steps in pseudo code and then prompt guiding an LLM through figuring out the required code often works great for smaller tasks.
I've also learned many new tricks by just asking it why it made certain choices.
Not really, do normal people assume that they're important enough to be the only Their Name of note in the world that their opinions would be known.
Absurd level of narcissism.
The whole Meta properties are turning into a similar mess. Facebook started to look like MySpace, which died because it became super slow and unpleasant to use!
Most teenagers don't want to touch Facebook or Instagram — the latter being used only by girls with severe narcissistic disorders. Most kids blatantly tell me Facebook is what old folks use. They mostly use Discord nowadays, not WhatsApp.