Hacker News Reader

Comment by thrwayaistartup | Hacker News Reader

Looking back in 25 years, the "Hallucination Problem" will sound a lot like the "Frame Problem" of the 1970s.

Looking back, it's a bit absurd to say that GOFAI would've got to AGI if only the Frame Problem could be solved. But the important point is why that sounds so absurd.

It doesn't sound absurd because we found out that the frame problem can't be solved; that's beside the point.

It also doesn't sound absurd because we found out that solving the frame problem isn't the key to GOFAI-based AGI. That's also beside the point.

It sounds absurd because the conjecture itself is... just funny. It's almost goofy, looking back, how people thought about AGI.

Hallucination is the Frame Problem of the 2023 AI Summer. Looking back from the other side of the next Winter, the whole thing will seem a bit goofy.

View on HN

My feeling is that GOFAI had a real problem with representing uncertainty, and handling contradiction. So, we tried to approach it theoretically, with fuzzy logic and probability and so on. But the theoretical research on uncertainty didn't reach any clear conclusion.

Meanwhile, the neural nets (and ML) researchers just trucked on, with more compute power, and pretty much ignored any theoretical issues with uncertainty. And surprisingly, with lots of amazing results.

But now they hit the same wall, we don't actually understand how to do reasoning with uncertainty correctly. LLMs seem to solve this by "just mimic reasoning that humans do". Except because we lack a good theory of reasoning, it can't tell when mimicking is bad and when it's good, unless there is a lot of specific examples. So in the most egregious cases, we get hallucinations but have no clue how to avoid them.

I think that ascribes way too much meaning to hallucinations, which are the artifact of a big fancy markov chain doing what you'd expect a big fancy markov chain to do.

I don't get your argument about the frame problem. Maybe it's like squeezing a big pillow inside a small bag. A bulge forms that won't fit. It's the frame problem. Turn the pillow around, squeeze it into the bag again: a bulge now forms on the opposite side: it's the hallucination problem. I can see how one could be the solution to the other. Hallucinations as a lack of rules.

That's an excellent summary I have to say. Theorists pushed hard to move the needle and practitioners with immense computing power reached and started chipping away at the same wall.

LLMs transpose the problem by mimicing what humans would do

I don't know much about AI research but the idea of "measuring" hallucination definitely seems very loaded to me. Humans hallucinate too and I don't think we can measure that. It almost feels like "we need AGI in order to develop AGI".

Comparing human hallucinations with model “hallucinations” does not make sense to me.

Model hallucinations seems to me like a fancy way to call the model results that make no sense (ie blatant errors). Plus it makes the model more humanoid.

Humans also very often produce results that don't make sense.

Humans that produce output like LLMs are most likely to be diagnosed as schizophrenic, which I don't believe is the goal.

I really haven’t seen much of that coming from my ChatGPT usage, it’s just someone lying with a lot of confidence, hardly a mental disorder.

Confident human bullshitters seem to thrive in business environments, in media and entertainment, in politics ... in fact in any profession where the production is just language instead of doing things. They might be more on the dark triangle spectrum, but I would not call them all "schizophrenic".

The problem is we are so used to yielding to confidence we don't apply the necessary checks even when we know it is projected by machine.

Untuned LLMs are most like people with Korsakoff's syndrome. "hallucination" is a misleading term.

Most hallucinations make sense. In fact, that is precisely the problem. They make so much sense it's often difficult to distinguish. Most people refer to hallucinations as wrong and often confidently wrong details in a generated reply.

Humans are certainly better but we don't have an absolute sense of what we do or don't know either.

Does anyone think we would have AGI if only we could solve the hallucination problem?

There’s people who thought we could just wire up ChatGPT to a bunch of API calls and have AGI by now. Or some similar version of bootstrapping an LLM.

That could very well be the case.

No, because there are still other issues like context length and performance. AGI is not very useless if you get one token per hour, and when it forgets about what you talk about 10 minutes ago because it ran out of context

We don't even have a generally accepted definition of AGI yet, so...no.

"... .Looking back from the other side of the next Winter, the whole thing will seem a bit goofy."

for most of us, what we wish for is what we believe.

Is from ought.

Reply on news.ycombinator.com