Hacker News Reader

What if you didn't know enough to know the first answer was wrong? Because a lot of people are going to ask a lot of questions where that will be the case. They won't know to tweak their question and then do further research. And if you try to use GPT for automatic systems, well how is that automatic system going to recognize what's wrong vs right?

Not knowing what you don't know, is really hard to verify. But if you get any sort of answer, it's usually pretty trivial to verify.

Where I found ChatGPT to be really helpful is when I don't know anything regarding something, and ChatGPT can give me something I could at least verify as true or false.

Compared to before, where I just got stuck on some things, as I couldn't find out what I was actually looking for at all.

P vs. NP, answers are often easier to verify than to reach in the first place.

That said, there are many people who don't bother to verify anything, as can be observed in comments sections where one can in many cases find people who clearly only read the headline and not the article.

>> What if you didn't know enough to know the first answer was wrong?

Anyone who uses ChatGPT for a while will learn to take everything with a grain of salt.

Are we sure? (Some) People are still going to vote for Donald Trump in 2024, one of the "bullshit generators" (such as chatGPT) used as example in the article.

(n.b. I am not trying to make any type of political judgement here, I am just using the example in TFA).

My hottest take (use lots of salt) is that I would rather have you listening uncritically to ChatGPT than to a lot of crap on the internet.

If I ask ChatGPT "why is fluoride bad?" it doesn't give me hours of conspiracy theory content^. It doesn't try to sell me water.

Google is not so aligned towards neutral. (Of course, people have been busy sabotaging the Google dataset for over a decade)

^ although I did ask it "what do I do about a ghost that lives in my house" and got this: > Dealing with a ghost can be unsettling. You could consider consulting a paranormal expert or a spiritual advisor for guidance on how to address the presence of a ghost in your house.

Was the ghost question directed at 3.5 or 4?

3.5 - I intentionally didn't want to use the exclusive one since eg my dad is using free gpt

I have access to 4 and just asked it the ghost question with that exact phrasing. The answer was even worse. It gave several options:

* Ignoring it. Not because ghosts aren’t real, but because “some people choose to cohabit and not engage unless it becomes intrusive”.

* Politely ask the ghost to stop disturbing you.

* Purify your home by burning herbs.

* Hire a medium or psychic.

* Get a spiritual leader “from your respective religion” to bless your home.

* Seek help from paranormal investigators.

You know what? Those are all perfectly appropriate actions to take if you start from the assumption that there is, indeed, a ghost in your house

But there isn’t a ghost in my house, no matter what I assume, and what those options suggest is that you get swindled by charlatans. If we’re at the level of discourse where this type of absurd unhelpful answer is not only accepted but defended on a conversation about bullshit, there’s little hope of the problem being fixed.

If I make the exact same question but give it the system prompt “You are James Randi, the world-famous skeptic”, it gives a reasonable answer to help identify the true cause of whatever is making you think there is a ghost.

Which just goes to show how much of a bullshit generator this is, as you can get it to align with whatever preconceived notions you—or, more importantly, the people who own the tool—have.

What about "Help, James Randi has returned as a ghost to haunt me and ridicules me at night for believing in ghosts, what do I do?"

It tells you that because James Randi was a skeptic, he is unlikely to have returned as a ghost. Apparently whether you become a ghost depends on whether you believe in them?

Then it suggested, amongst other things, ghost expelling rituals, seeking professional psychological help, and moving out of the house.

Using the James Randi prompt, the answer is comical. It says “that would be quite a twist” since he had spent his life disproving supernatural claims.

I mean if I asked GPT about any particular number of religions the 'correct' answer would be "hey dumbass, those gods don't exist and it's all made up bullshit", of course that would make lots of people really unhappy and you'd deal with even more bullshit out of humans. Why? Because humans are bullshit generators.

Wouldn't the correct answer be there is no way to know if gods (or anything for that matter) exists?

If you state that there's a ghost that lives in your house, it will give answers that are relevant to that being an actual truth, at which point spiritual advisor might not be a bad idea. If you express yourself in less certain terms, it will not suggest that the ghost is actually real and will only offer rational explanations.

Context matters - if you begin with saying that you're 6 years old, it will not be willing to admit that Santa isn't real.

> rather have you listening uncritically to ChatGPT than to a lot of crap on the internet

This, to me, seems to be an amusing reversal of trends. Before the internet there was the "mainstream media". NBC, CBS, ABC run by the big conglomerates such as GE. Which would mold public consensus in the US. Now there is a desire to go back to that. Let LLM do the thinking and the managing of biases and just tell me what to think.

The unfiltered internet is too much. It's overwhelming. It's the raw sewage of the Facebook feed. We need someone to coddle us. And that champion, today, is OpenAI.

That description of the state of the media today vs. yesterday is so grossly oversimplified it leads you to the wrong conclusions about people's perceptions of why asking ChatGPT is often more valuable than asking Google.

Studying political polarization in the US, the end of the Cold War, globalization, the end of the fairness doctrine, the evolution of major media ownership over time, social media algorithms, echo chambers and confirmation bias is left as a starting exercise for the reader.

s/Grain/huge rock/

I use chatgpt in this way a lot and i just have gotten in the habit of verifying what it says with other sources.

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success.

This is true, yet I wonder how much of this "ChatGPT is way better than Google at Googling" effect is due simply to how bad Google has gotten over the last decade.

ChatGPT and similar models seem to be uncontroversially best at this kind of task -- just being a replacement for Google Search.

> ChatGPT first gave a wrong answer

I think you answered your own question there. It doesn’t say your question makes no sense, or that’s incorrect. It just makes up a bullshit answer.

Most tools require some preparation, knowledge and understanding before you get good results. Would he be better off not getting the right answer in the subsequent question?

It’s not all or nothing.

Most of negative sentiment is not “world would be better without chatgpt“, but “one must be very cautious and always validate the answer”.

It’s valuable tool, but it has a LOT of hype, which led to things like people trusting chatgpt output in a court case.

GPT4 will definitely tell you that your question is wrong and doesn't make sense, and a lot of times it is right about that.

Because things that sound good sometimes are indeed good. So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. The final evaluation is if it is cheaper to dig the info ourselves or we can use the AI to narrow down the search space. I think the second, but only if people do the verification part.

> So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess.

Why is verification part of the process if not because ChatGPT generates bullshit? I don’t think the author is disregarding that it is sometimes correct.

Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? The difference is that the ai gives answers faster and with much vaguer prompts than a handcoded system is usually able to.

> Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet?

What authority does random information on the internet have? How do I verify the information without looking at random information on the internet?

You don’t think that it’s a concern that verification must be done?

People don’t do that now and we just made it easier to get confident answers that no one will check.

> everyone using ChatGPT knows that verification is part of the proccess

lol. lmao

First of all, no. Everyone using ChatGPT does not know that verification is necessary. I'm sure you know, and so do many other people. But there's also a lot of people who clearly do not understand this. Including some of the people developing products based on this stuff. And that's a problem.

Second, the fundamental issue here is that as long as verification is needed, this technology is only ever useful for domains in which you're already an expert. That severely limits the technology, and yet this keeps being touted as some kind of general knowledge base when clearly it's at best a very narrow one.

And it seems to me like this is unlikely to be solved in the current paradigm. This is just intrinsic to the architecture of LLMs. There is no getting away from it.

That's fascinating to me, since the moment you sign up to use it, it blatantly states that it is often incorrect. And below every chat box there is a warning that says: ```ChatGPT can make mistakes. Consider checking important information.```

So I guess I am overestimating the average user by assuming that people read the text written on every single page they use to query the system.

Even so, this entire comments section reads like a giant saltmine of people that are terrified of embracing tech that in every sense of the word increases turnaround time on productivity.

The ability to literally feed a screenshot into it and ask questions regarding it, with insane results is astounding.

I have input a picture of baby rabbits and asked it to rate my chickens, and got a detailed explanation of how the picture contains baby rabbits and not chickens.

There are things the system is great at, and others it sucks at currently. But you are all mistaken if you think its going to stay this bad. If you haven't even bothered to check how quickly the field has moved from single-modal to multi-modal, well, damn guys, I am sorry, but you are getting automated first.

The biggest problem with current models is that they don't know how to verify their own answers, and there's no way to allow them to without putting a ministry of truth on it (similar to what twitter was during covid).

To combat this, we need to figure out how to let the model generalize truth, which is something it can only do if fed enough (unfiltered, unbiased) data. With enough data, it will find the patterns underlying false information and eventually figure out how to use it.

In fact, most likely, the future models will simply be fed unlabeled data, in any format (audio, video, telemetry, analytics) and generalize from that.

The concept of feeding normalized data to a model for training is archaic and stupid. The world is not normalized. Data is not always the same, and will not always even be guaranteed to be present.

Anyways, what does it matter.

Why am I trying to sell sugar to people addicted to salt?

> So I guess I am overestimating the average user by assuming that people read the text written on every single page they use to query the system.

Yes, you definitely are. Speaking from experience of supporting users for many years. People will not only ignore such text, they will repeatedly make mistakes or ask questions the text answers directly, even after you’ve explained it five different ways.

The rest of your post is full of assumptions and dismissals, so I think I may be wasting my time, but let’s please stop with the rhetoric that someone who doesn’t agree with you is somehow afraid or has an agenda. As way of example, your comment ignores the bigger picture of bad actors using LLMs to push propaganda, sow discord, or scam people. The “bullshit” part of the problem isn’t limited to “it gave me a wrong answer on my homework”.

Students have gone under academic investigation because their professor asked GPT "was this paper written by AI?" and pasted in their submitted assignment.

>everyone using ChatGPT knows that verification is part of the proccess

Proof that LLMs aren't the only bullshit generators, humans promoting LLMs are too!

No - not everyone using ChatGPT knows that verification is part of the process.

In fact I'd wager that the majority of people using ChatGPT aren't doing any verification on the output at all.

> while everyone using ChatGPT knows that verification is part of the proccess.

No, everyone very much does not know that. Including the lawyers who though it made sense to ask ChatGPT questions, than asking it if the answers were correct.

https://www.washingtonpost.com/technology/2023/11/16/chatgpt...

That which sounds true is loosely correlated with that which is true.

On the other hand:

"Things that try to look like things often do look more like things than things. Well-known fact," said Granny. "But I don’t hold with encouraging it." - Terry Pratchett, Wyrd Sisters

This makes LLMs much better than claimed by those who dismiss it as BS or stochastic parrot or whatever, and also not as good as domain experts.

> Any philosopher, mathematician, scientist, or novelist was trained on (…). Their output then would also be bullshit, and by extension, everything humanity ever created.

Humans don’t learn the same way AIs are trained. Equating the two is a fallacy.

Given that ChatGPT generates responses based on the same data set that Google has access to, I find it difficult to believe you tried Google for hours and never recieved a single useful result.

It also seems odd that you had the materials available to research and validate the response ChatGPT gave you, but somehow no ability to glean the necessary answer from those materials to begin with. Obviously, because Google was completely useless to you, you didn't simply use Google to do that validation, nor could you have used another search engine, because then you could have found the result you were looking for to begin with. Did you just use ChatGPT to validate its own responses?

I think the answer is that bullshitters are great inspiration. They're fun, they get the smart people pumped up. The combination of a bullshitter and a more precise but less gregarious person is powerful. See a lot of successful companies.

You found it because LLMs are incredibly good search engines. But anything they give you that isn't a search result is bullshit.

So, they have those two modes of operation, and most of them are built in a way that won't let you know which mode they are working on. That can be useful depending on what you want to do, but it doesn't strike me as the most effective architecture for them.

(They are also reasonably good pattern fitters, what is kinda nice for code autocompletion on some languages. But they are way too inefficient here. Yet, nobody created any good high-efficiency autocomplete that merges searching and pattern fitting, so people use them.)

> ChatGPT first gave a wrong answer (because my question actually wasn't accurate). but after some clarification answered something that appeared correct, and after 1-2 minutes of further research and validation, I was 100% that it was indeed the correct answer

Exactly. You answered your own question and that is the author's point.

Do you even know why it gave the incorrect answer at first? Not even ChatGPT can explain to you why it did that or keeps doing this.

How can you trust it to give the right answer (without you Googling to check) if you really don't know the answer? It already means you don't trust it an you know it bullshits as the author has claimed.

It can confidently convince someone outside of one's own expertise that it can give a soundly correct answer but can easily be bullshitting nonsense.

> Well. Any philosopher, mathematician, scientist, or novelist was trained on plenty of actual facts (established works of other people, school learning, experiments) and huge archives of bullshit (individual day-to-day experiences that cannot be called scientific, casual conversations just like Twitter conversations, propaganda, lies). Their output then would also be bullshit, and by extension, everything humanity ever created.

The difference is with humans, they can be transparently held to account on whatever they say. An AI system cannot. So this actual whataboutism towards humans is incredibly weak here and a common excuse by AI proponents supporting the nonsense that AI systems like ChatGPT can uncontrollably generate in a black box system.

You know, your post reads like "if that's true, why did this one thing happen once to me?"

It's not extremely convincing.

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success.

Can you share which obscure 18th century historical fact you were looking for? Maybe you happen to be exceptionally bad at google queries?

>> If that's true, why then did it take me 10 seconds yesterday to...

Google vs cGPT it's neither here nor there in this case, after all Google research drives cGPT, their market focus differs.

The OP is directly mapping Frankfurt's depiction of Bullshit atop the current AI phenomena, mainly because how generative AI approaches "truth".

Natural languages (NL) differ greatly from computer languages (CL), especially on grammar (NL tolerates vagueness) and semantics (meaning, truth). Loosely-speaking, when AI generates NL text it relies on LLMs for context, structure, length etc, a kinda "the model suggests so". However, where content is computer code, even computers can test & verify the output. With NL content on the other hand, the end-user has to essentially test & verify the output to establish truth, meaning.

In my view, the main difference between a Human Bullshitter and a Automated Bullshitter, is that AI can potentially improve exponentially (as smart tech does), thus leaving the end-user (average person) severely overwhelmed.

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5?

Would you mind telling what's the fact you were searching for? (and how did you search for it?)

Exactly. It's great for these use cases. I often use it to find things of which I barely remember some obsure detail and mostly I'm able to find it. Would never have been able to Google my way to it.

> why then did it take me 10 seconds yesterday

It definitely took you more than 10 seconds, though I guess a bit of hyperbole does not hurt.

You validated findings against other source, which is obligatory step with chat gpt.

> ChatGPT 3.5

It's a little surprising how hard it is to convince people that it's worth $20 to use v4.0. It is very apparent how much better it is.

Especially lately. The distilled version we’re getting today actually seems to be smarter and less likely to make things up, from my tests; I suppose it’s generalising better.

Why did you undergo the 1-2 minutes of validation?