Hacker News Reader

Comment by fenomas | Hacker News Reader

It's so weird to me when apparently tech-savvy people say ChatGPT "lies", or call it a bullshit generator and so on.

I mean - when you ask StableDiffusion to draw a dog astronaut, everyone gets that the image it returns is made-up, right? Nobody expects the AI to return only "true" images of existing things - it was trained on fictional images as well as photographs, and people understand that it can imagine new things beyond what it's seen. Nobody expects SD to emit an error like "I can't draw a dog astronaut because they don't exist".

So why do people expect ChatGPT to work differently? Even with developers who presumably understand the technical details, I constantly see people acting as if it was an error mode for ChatGPT to say something that isn't factually true about the world. How is that any different from calling SD a liar because it drew a dog astronaut?

View on HN

Because it is conversational and appears to answers direct questions.

The image generators accept a prompt, not a question, so we don't expect an answer.

ChatGPT generates responses to prompts that make it sound like it is answering a question that is posed.

Ask ChatGPT a question. The mere fact that I can fairly reasonably pose that as "asking it a question" is why people get confused. It's very easy to interact with it conversationally, and weird to interact with it otherwise because the text it generates is conversational. A generated image is never a conversation, so of course it isn't an answer and so can't be wrong.

I think this is a big part of it, yeah - I don't remember people calling earlier GPT models dishonest, when the UI presentation was of the AI returning a continuation of the input prompt.

I do tire of these appeal to hypocrisy/inconsistency type arguments that rely on collapsing important distinctions.

It’s generally accepted that artwork can delve into the fantastical and absurd. That’s part of the point of it being art.

ChatGPT isn’t for the most part trying to be performance artwork - it is striving to be ‘right’ and to give the correct answer to your question.

I don't think that premise stands. If I ask ChatGPT to write a poem about Sherlock Holmes fighting a mermaid it will happily do so, and I think most users would agree that it'd be a failure mode if it refused.

Isn't it more the case that people are expecting it to sometimes be an artist and other times be a true-fact database, and assuming the AI will know which one they want?

No, because inherent in them asking ChatGPT to write them a poem about Sherlock Holmes is that they’re asking ChatGPT to write something fictional and artistic.

That's what I just said - people are expecting the AI to discern from the prompt whether to answer in fiction mode or fact mode. And my point here is, people who understand the tech should know that only one of those modes actually exists! It's a generative AI; it doesn't have a fact-retrieval-mode any more than StableDiffusion does.

I mean this is a bit of an apples to oranges comparison isn't it...

On one hand, you're talking about how AI produces artwork from prompts, where you're expecting the output to be made up/fictional.

And on the other hand you're talking about ChatGPT which is something that a lot of laymen are looking like a replacement for tons of things like copywriters, software engineers, Google search, etc. Every single one of these things have a pretty high requirement for the output to be at least pretty close to accurate. If anything I don't think a lot of the people who have an issue with ChatGPT (myself included) actually have an issue with ChatGPT itself, but rather with the tons of people, laymen and otherwise who have pointed to it as the metaphorical AI singularity when in reality it is little more than a iteration on existing AI models packaged in a better form for people to understand it.

That's how expectations work.

When you have been training people to expect realism and the truth through chatbots (Siri, Google Assistant, even the various bots on commerce/service sites), that is what they will also expect from ChatGPT which is represented in the same exact way. This itself is built on messaging expectations, if we're talking with our friends and family we don't expect them to suddenly start making everything up. We expect the truth, or at least their truth.

Compare this to art generation, say Dalle 2, which instantly showcases various images and highlights the wacky ones. This builds on previous expectations people have of art in general.

This is a crucial lesson in how presentation matters. Give ChatGPT a silly mascot, make it always output an informal lowercase internet-slang tone, you'll quickly see how expectations change.

Whether something is an error mode doesn't depend on the design of the tech, it depends on the use case it's being applied to.

I frequently point out that GPT generates falsehoods because I'm constantly reading stuff from people who try to use ChatGPT as a knowledge retrieval engine. In that context, generating bullshit is an error mode, and the correct answer isn't to try to fix ChatGPT, it's to raise awareness of its inherent limitations.

So, I agree with you wholeheartedly that generating bullshit is a feature, not a bug. I just want the rest of the world to realize that.

Your analogy would only hold if we were asking ChatGPT to write poetry, or descriptive fiction.

But its method of presentation is one where it on the whole composes paragraphs composed of proposed facts and logic.

There's really no ambiguity here. Its sentences are in general (not all sentences, but most) composed of propositions. The propositions can be tested for truth or falsehood. If the proposition is false, we can generally call it a "lie". Or at least -- not a fact.

We expect it to work differently because conversational language works different than images and art.

When stable diffusion is told to make a dog, it's fairly obvious when you get one of the AI generated monstrosities.. and people discard those easily.

If you ask ChatGPT if Ebola is transmitted by mosquitos, and it says it's not only transmitted by mosquitos, it's airborne and highly contagious... do you know if this is accurate? Most people don't ask Stable diffusion to make an images of things we don't understand, while we ask ChatGPT questions we do not know the answers to.

I think a fairer analogy is always expecting a Google search to return factual, objective, and high quality information. Similar to Chat GPT, the results are often mixed and depend on the way you structure your search, the content area, etc.

Do you mean dog astronaut as an example of a lie? Because there have been dog astronauts.

Fair point ;) Yes, I meant it as a "cow riding a motorcycle" kind of example - concepts the AI knows about in a configuration it hasn't seen.

One generates art.

The other speaks authoritatively about falsifiable facts and is frequently wrong.

Art is subjective.

Facts are, in theory, not.

Reply on news.ycombinator.com