Hacker News Reader

Comment by jedberg | Hacker News Reader

Of course not. Humans "hallucinate" all the time too.

Why do we hold AI to a higher standard than humans? It's the same with self driving cars. "Oh it had one accident, must not be safe!" and yet humans have 100s of accidents a day.

The main problem here is expectations -- everyone expects the machine to be perfect, and when it's not, it breaks expectations. In the past, machines were generally a lot more accurate, they just didn't do a lot of stuff. Now they they can do a lot more stuff, their accuracy is coming down to human levels, and it throws everyone off.

I don't think we need to fix hallucinations, we need to fix expectations.

Humans don't have 100% perfect recall of every fact they've ever learned, why do we expect AI to?

View on HN

Because we're building something that runs autonomously at scale. Similar reason why you don't give one human complete power over the entire economy without mountains of trials/elections etc.

Expecting more from AI is being cautious about wanton stereotype machines that will remove plenty of "redundant" humans from the workforce without demonstrating that they are capable of redundancy against error.

If we're digesting and calcifying human knowledge, it's likely we have only one shot to get it right, so we had better do it correctly.

Ask yourself if you would have been fine with calculators if they did math at an equal error rate to humans.

Because a lot of the time, AI is _pitched_ (and hyped) as a higher standard than humans.

That is true, and a big part of the problem. Vendors need to stop pitching it that way.

> we need to fix expectations

Properly calibration expectations is part of it, but surely we could do better by having the AI cite proper sources and having it show us the line of reasoning? The problem is that right now you ask an LLM a question and it'll confidently give you an incorrect answer without any sources or explanation of the reasoning behind it.

Having tools that shift blame and confidently lie to you is a problem. Here's a recent example that bothered me: I tried using Meta AI to generate an image and it failed. When I asked why it failed, it kept lying and making up excuses for how it was my fault or my browser was doing something wrong. When in reality the probable explanation was that the image prompt included something that was probably flagged as sensitive content, which is why the image generation failed. (It wasn't even anything nefarious, I just tried several variations of catgirls.) If something goes against some content guidelines I want the tool to tell me that, rather than gaslight me with lies.

In most human interactions, do people site their sources? If someone doesn't want to tell you something because it makes them uncomfortable, do they tell you that or do they lie to you to make you feel better?

I'm with you, it would be nice if LLMs could do those things, but also I don't expect my human friends to do that, so I'm not sure why I should expect it of my AI.

I'm afraid hallucinations would be easier to fix than expectations. We are humans after all, right?

We generally send halllucinating humans to the hospital.

You must be kidding. Every day my spouse and I have a conversation that involves one of us realizing that we've mis-remembered something. Not getting every fact correct is super-common in humans.

This right here is a perfect example of the problem of expectations. Do you have 100% accurate recall of every fact you've ever learned?

Then why do I need a computer to mis-remember things for me.

Maybe you don't. A lot of people seem to find it useful though.

Everyone hallucinates stuff, you don't need to see the Virgin Mary to hallucinate. You hallucinate stuff every day, and you are not even aware of it. Even had a "déjà vu" ? Saw your black cat in a towel in a dark room ? Every day.

Persistent and frequent/high-percentage hallucinations? Yes, that's cause for concern/medical consultation.

But we don't over-react when humans are fooled by optical illusions/magic tricks, or when they conflate two things, or when they confuse correlation and causation, or when they selectively prioritize some information over other information, or...

> Persistent and frequent/high-percentage hallucinations?

This describes the current state of AI, doesn't it?

If you're determined to find a reason to negatively judge genAI, you can find examples to fit that narrative.

I think that it does not happen at a level of persistence/pervasiveness that would cause the equivalent human to be hospitalized.

Reply on news.ycombinator.com