Hacker News Reader

Comment by zht | Hacker News Reader

Are you able to tell when it is hallucinating?

Not right away, which is why I prefer bing over chatGPT (less tendency to get things subtly wrong and gaslight me). However I see no significant distinction between a hallucinating LLM and a person who made a mistake. It's something that the LLM engineers and the LLMs will work on every day. There are definitely some caveats but I maintain that less energy is required and I get faster and more tuned results than an SO post or trawl.

Are you able to tell when a human is?

If it is a community effort yes. Yes we are. That’s the idea of SO. It’s a community effort. Highly flawed and elitist yes, but the theory is it is community policed.

If that is good enough, you can replace the "community" with a cohort of different LLMs, which should work analogously.

No? Maybe if they reach human levels of sensory and cognition. For example, again, my FairPlay problem. On iOS 16 there is a timing issue when DRM can be ingested, but this isn’t documented from apple and it is kind of a rare problem. If I ask Bing right now, which is connected to the internet. It gives you links to a human stackoverflow. How would an LLM know about this timing issue? The source code looks fine, there is no compiler error, there are no crashes. There IS sometimes just a video that is not playable in AVPlayer.

How does an LLM with an LLM community solve that?

unfortunately the way web search is implemented in bing chat kind of hamstrings it from that point forward. You miss out the LLM extrapolating from a million other scenarios accross the subject domain in order to come up with possible explanations or approaches. Likewise it is not searching every page multiple adjascent topics to find something on the web that is deeply burried or going by another name because of resource constraints.

It’s really amazing how low our bar has become

Reply on news.ycombinator.com