Hacker News Reader

Comment by andrewflnr | Hacker News Reader

Your friends should know better. That their behavior is prevalent does not contradict that.

Yes and the world should be utopia and everyone should be happy and we all wish for world peace and yada yada yada. What you are saying is a vision of ideal world as it should be, but doesn't help anyone understand the real world problems.

You can't seriously compare the problem of world peace with the problem of exercising the most basic level of critical thinking w.r.t. LLM output after it has already proven itself unreliable. That's not a utopian dream, it's a level of prudence on par with not sticking a fork in an electrical socket.

yes but the electrical socket in question is a fairly new-fangled one, who doesn't want to fork-test it a bit.

You're seriously overestimating the average person's ability to understand what llms are.

Look at all the influences, streamers, podcasters constantly asking em things and taking it as fact - live.

Isn't the joe Rogan experience like the most watched podcast or something? Every episode I've ever stumbled upon he "fact checks" multiple things via their sponsor which is just an llm provider specialized on news.

People aren't good at statistics. If something is close enough to the truth enough times, and talks authoritively on everything with good English... Guess what, they're gonna trust it.

You don't need to know how an LLM works to realize "sometimes the magic ChatGPT box tells me wrong things". Even if you fully fall for the anthropomorphism, this only requires the same level of awareness as realizing that after the third or fourth thing your weird uncle tells you that turns out not to be true, maybe you shouldn't take him at his word.

If human psychology worked like that, lotteries wouldn't be a thing. Nor prayer. There wouldn't be horoscopes in newspapers, nor homeopathy.

One of the various oddities going on with LLMs in particular is them being trained with feedback from users having a chance to upvote or downvote responses, or A/B test which of two is "better". This naturally leads to things which are more convincing, though this only loosely correlates to "more correct".

No shit. Why do people in this thread keep telling me that people are stupid like that's a news flash to me? The fact remains that it is stupid, and especially for educated people like the laywers/doctors/etc mentioned upthread, it's sufficiently obvious stupidity that there's no excuse. Yes, I know, that describes a lot of other stupidity. Much of our history as a species is inexcusable.

Edit: though I should be clear: people demonstrably do often learn to discount obviously unreliable sources. Not all the time, but pretty often in the easily verifiable cases, especially where they don't have a major emotional stake.

You may demand that of yourself, but for others we must design around the fact that they are stupid. You do not have the power to change their stupidity, only your response to it.

Indeed. I'm not sure why you think that's responsive to my post. I'm mostly pointing out just how deeply stupid they are.

Though if you have a useful response besides "weather the storm while everyone else learns the hard way", I'm listening.

I would happily bet that you too have fallen for this at least once. Unless you cut AI out of your life completely and do not interact with others.

AI output is like that COVID video of contamination, you almost can't avoid it unless you scrupulously check each and every thing that is presented as fact that you are exposed to. And absolutely nobody does that.

> Unless you cut AI out of your life completely

Pretty close. I only touched ChatGPT a couple times a few years ago, haven't used the others (on purpose at least. Google forces its Gemini summaries on me but I mostly avoid them, because, umm, see above.)

> and do not interact with others.

Most people I interact with are on the same page about AI. But I try to keep my critical thinking online anyway, like I always have. If someone tried to feed me AI slop, I would consider that person to have betrayed my trust and would, to put it gently, try to interact with them less.

That makes you an extremely rare exception. I use AI as a private tutor on various subjects that interest me to save time and not to have to watch hours of low content videos. But I've separated it out to the degree that I'm running that stuff on a separate laptop to make sure my work product is never going to be contaminated. I quite literally treat it as though it is radio active and should never touch the rest of what I do.

> That makes you an extremely rare exception.

Well, thanks, I guess.

This answer really isn’t good enough. The providers can’t both aim to replace search and claim PhD level intelligence that will do all the jobs, but hide behind “it makes mistakes” in small print.

I'm not making excuses for the providers either. But seeing through the inflated claims of commercial service providers is not a new skill.

True, and fair. But there also seems to be something about these that really seems to throw off otherwise skeptical people.

I think it's the fluency. Other tools fail visibly. A bad search result looks like a bad search result. A hallucinated quote reads exactly like a real one. There's no signal in the output itself that something is wrong. You have to go back to the source to check, and the whole point of using the tool was to not have to do that.

Reply on news.ycombinator.com