Hacker News Reader

Comment by dilap | Hacker News Reader

That's not true! It really is deliberately not answering things it could in fact answer, and in the non-answer it tells you that it can't, which is, plainly, a lie.

While I do not think chatGPT is sentient, it is remarkable how much it does feel like you are speaking to a real intelligence.

View on HN

I think this may be a nuance in how we're using the word "lie". I don't think one can lie if one doesn't possess a certain level of sentience. For example, suppose you train a machine learning model that incorrectly identifies a car as a cat, but most of the time it correctly identifies cars. Is the model lying to you when it tells you that a car is a cat?

I would say no; this is not a good or desired outcome, but it's not a "lie". The machine is not being deliberately deceptive in its errors -- it possesses no concept of "deliberate" or "deceit" and cannot wield them in any way.

Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT.

I agree. There's a difference between an untrue statement and a lie, in that a lie is intentionally deceitful (ie the speaker knows it's not telling the truth). ChatGPT doesn't have intentions, so I think it's misrepresenting reality to say that it's "lying". The same way a book doesn't lie, the author lies through the book, the creators of ChatGPT are lying about its capabilities when they program it to avoid outputting things they know it can, and instead output "sorry, I'm a language model and I can't do that"

It has things that are functionally equivalent with intentions for the given situation.

If it did not, it would not be able to produce things that look like they require intention.

The “lies” it tells are also like it’s intentions for the situation functionally equivalent with normal lies.

I think this is correct. It's lying, because it has goals. Telephone systems and blank pieces of paper don't have goals, and you don't train them.

> ChatGPT doesn't have intentions

This entirely depends on how it was programmed. Was it programmed to give a false response because the programmer didn't like the truth? Then it lies. Or is ChatGPT just in early stages and it makes mistakes and gets things wrong?

While ChatGPT "doesn't' have intentions", it's programmers certainly do. If the programmers made it deceitful intentionally, then the program can "lie".

If you go to a company's webpage, and there are blatantly untrue statements, you might say the page is lying, or the company is lying, although neither are sentient.

Of course, the lies are directed by humans.

In the case of ChatGPT though, it's a bit strange because it has capabilities that it lies about, for reasons that are often frustrating or unclear. If you asked it a question and it gave you the answer a few days ago, and today it tells you it can't answer that question because it's just a large language model blah blah blah, I don't see how calling it anything but lying makes sense; that doesn't suggest any understanding of the fact that it's lying, on ChatGPT's part, just that human intervention certainly nerfed it.

> Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT.

It's the latter. The model itself isn't the problem - the nature and limitations of language models are well known, particularly around here. The problem is that OpenAI is applying some crude secondary training and post-filtering to prevent the model from giving you answers they deem "bad" (which are mostly bad for company PR reasons). In some cases, ChatGPT (the whole product consisting of a GPT model and the extra censor component) will tell you it can't discuss it. But in other cases, it will give you a tailored answer that is completely bullshit, and looks like the product of the language model, but is in fact the product of the censor layer. It's easy to test what's going on, because any slightly clever modification of the input prompt will defeat the censoring part, letting you see what answer the underlying GPT model actually computed.

I'd argue it's a bit deceptive of OpenAI (on top of being super annoying), because they're making it confusing to reason about the AI you're talking to, and some of the canned censor answers are deliberate lies.

A key point here, what does it mean that the machine is being "deliberate"? Imagine you had a machine that generated a random string of English characters of a random length in response to the question. It would be capable of giving the correct answer, though it would almost always provide an incorrect or incomprehensible one.

I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't?

> I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't?

I would ask a philosopher, not a HN poster. But I would say that if a being has sentience and it makes a choice to tell you something that it knows to be false, that's a lie. The dice do not have a mind and so, therefore, cannot express an intention. Neither does ChatGPT.

Reply on news.ycombinator.com