Seriously Embarrassing Hours for AI

garymarcus.substack.com

5 points·jdkee··4 comments

It's a red flag to me when "AI people" complain about what is now apparently called "hallucination" in LLMs. This is not a bug - it happens by design and is expected. Whether it's desired or not depends on the usecase, and while there are methods to steer hallucination to a certain extent, today's models cannot guarantee this because at their core, they still do nothing but estimate the likeliest next tokens. I don't expect any random non-expert user to be aware of this, but the amount of "experts" missing to understand and provide the full relevant context is disappointing.

"AI people" do understand that LLMs "hallucinate" by design. Rather, critics see hallucinations as a problem for applications that inherently require factual accuracy (e.g. Galactica should not hallucinate; summarization should be factual, etc.).

Indeed, one could make the case that LLMs are fundamentally unsuited for many of the tasks they are currently being marketed for.

"Galactica should not hallucinate" says who? I have no doubt its creators were very well aware of this property, also prior to releasing it. Unfortunately, they expected too much of the consumers, as is often the case with new technologies.

All of that is fine because that's just how revolutionary systems improve and align with society. My problem is the click bait type framing of the author without proper reflection.

Is this an AI generated article?

Reply on news.ycombinator.com