Hacker News Reader

Comment by timcobb | Hacker News Reader

This is all true for getting information from Google or other places. The same "critical reasoning" skills are required here, except the interaction is slightly different.

View on HN

The problem is the average worker is more likely to blindly trust an "advanced chatbot"(from a layperson pov) than a random google result.

(edit) how many people do you think are aware of LLM hallucinations as opposed to inaccurate google results

It's trivial to build some type of never-perfect verification on top of LLM output.

LLMs will be involved in complex tool chains like everything else.

> The problem is the average worker is more likely to blindly trust an "advanced chatbot"(from a layperson pov) than a random google result.

Why? Any who understands about Google or Wikipedia can understand about LLM.

This. I don't think people lower their thresholds for applying critical thinking / fact checking just because it is an LLM's answer.

If someone blindly trusts and uses an LLM without checking for more evidence, then they would've done the same with any other source, be it Google, the newspaper or whatever.

You know, like lawyers that present motions full of made-up case law in court?

"In a cringe-inducing court hearing, a lawyer who relied on A.I. to craft a motion full of made-up case law said he “did not comprehend” that the chat bot could lead him astray." [1]

[1] https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...

Yeah, lawyers can be negligent and lazy, too

I think you're giving too much credit to the average user, empirically, we as humans tend to question less and believe more about things that are beyond our understanding, and LLMs are certainly way more of a black box than a search engine. The idea of a search engine is intuitive enough, but a large language model which involves maths that people likely haven't even heard of? and is the next big thing(TM) and will revolutionise the world? Yeah I have a feeling it will take some time before the average user understands the ins and outs of this tech.

Also the problem isn't necessarily about understanding the tech, but more about how it will be perceived at first. As long as every big player hypes up the tech as if it's perfect and revolutionary, I can't blame people for blindly trusting it at first(because it might as well work for most things they use it for, it will take some time before the pitfalls become apparent).

If you query Wikipedia or Google for an hour on a topic you know about there’s much less of a chance you’ll get inaccurate information than with ChatGPT where it’s almost a given. Assuming people get some basic training it won’t be hard to ask someone in the class about a baseball record and then let them see ChatGPT hallucinate the players and dates.

Except, LLMs don’t cite their sources accurately (if at all). A Google search may yield information with a source I can verify; my understanding is that LLMs are fundamentally incapable of doing so.

If one gave you a 10,000 page bibliography would you read all the sources? The same problem exists (long bibliographies) in human knowledge transfer. I’m suggesting this is an architectural issue, not a LLM problem per se. LLMs are not supposed to be AGI.

I never need to read all the sources, only the sources that are relevant to the bits that I'm referencing, no?

If you only need bits of info with citation, an LLM can do that. It’s synthesis that’s the problem. It’s true for humans as well.

An actual citation and a string that happens to be formatted to look like a citation are very different. I wish we were at the point that most LLMs could do the former.

I've seen the former happen for some GitHub docs and when enabling plugins.

I agree there's cause for concern, and I hope it's a gradual rollout that's introspected in between each propagation of such, but I definitely think it's viable to be used no more less safely than other tooling in the government today (for better or worse).

An LLM backed by an embedding store absolutely backs its sources. It’s a solved problem. But people cling to the narrative because that takes the fear away.

SERPs have more cues about trustworthiness but yes it is a little bit like Google's knowledge graph results:

https://digitalreadymarketing.com/wp-content/uploads/2014/08...

Friendly reminder that critical thinking is dangerous. Just trust your betters.

https://www.nytimes.com/2021/02/18/opinion/fake-news-media-a...

Reply on news.ycombinator.com