Hacker News Reader

Comment by hospadar | Hacker News Reader

I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever - maybe it needs to learn when to say "I don't know the answer to that question".

That's a pretty hard thing for most humans (and myself) to learn to say, and I suspect GPT's training data (tha internet) doesn't include a lot of "I'm not sure" language and probably does include a lot of "I'm definitely sure and definitely totally correct" language (maybe, I guess, no evidence to back up that suggestion, I'm not sure).

Many of my favorite coworkers, friends, doctors, pundits are trustworthy exactly because they work hard to not profess knowledge they are unsure about. The reason (IMO) that Scott Alexander is a jewel of the internet is because of the way he quantifies uncertainty when working through a topic.

View on HN

It can’t learn because it isn’t intelligent. It’s a markov chain.

There is no way for it to make any decision at all. The fundamental system design is that it guesses the word that is most likely to be next. That’s it. There’s no concepts behind whatever it outputs, it has no idea what it means.

It's a next-word predictor like a Markov chain, but a Markov chain couldn't do all the things ChatGPT does. ChatGPT has learned lots of syntax-level patterns pretty well.

Is it actually a next-word predictor? I thought the training loss is against a set of words, not just one.

I'm not sure what distinction you're getting at, but transformers do use "fill in the missing word" training and text generation chooses the next word (token actually) one at a time. Once it chooses a word, it doesn't go back.

You’re just nitpicking. Everybody here knows that it isn’t intelligent. Words can mean more than one thing. And in this case the word “learn” is being used to describe what ChatGPT does to improve its answers over time. It’s perfectly reasonable to say that ChatGPT needs to lean to say “I don’t know”.

No, it's just anthropomorphizing.

>"I wonder if the biggest shortcoming of GPT right now is not that it sometimes gets things wrong, or can't cite its sources, or whatever "

It surprises me that people, here no less, seem to not realize that chatGPT has literally no idea what it's doing or saying. It's just trying to find the next word. It has no regard or understanding for what the word means. In response to this, people here just reflexively question what happens when a human is thinking, but I don't see that as a basis to assume that means chatGPT is thinking.

You're flat out incorrect about this. GPT loves to say that it doesn't know things. Have you actually used it?

Don't take my word for it. Ask it: "How many people are wearing tennis shoes in the United States right now?"

The response I got: "I'm sorry, I don't have the current data on the number of people in the United States who are wearing tennis shoes right now. It's not possible to determine this number as it would require constantly monitoring the footwear of every person in the country, which is not feasible. The number would also vary greatly from moment to moment and would be influenced by a number of factors such as time of day, weather, and location."

No hallucinations, no bullshitting. Just "I don't know" and an explanation of what it would take it find out. No idea why you think it has trouble expressing a lack of confidence in its answers.

I've actually used it quite a bit and he's not "flat out incorrect". I'd say you're the one wrong.

It might reply that when you ask it something that it can deduce it can't know, but ask it something reasonable and it will almost certainly hallucinate it instead of telling you it doesn't know about it.

> What function in Unreal Engine makes an actor transparent?

> The function to make an actor transparent in Unreal Engine is called "SetActorOpacity".

(There is no function called SetActorOpacity)

No, it's bullshit. Notice that the answer is not internally consistent. It first says that it does not have current data on that, which implies that it might have old data and that it would in principle be possible to generate that data. It then states that it is not possible to answer this question. Those two parts of the answer are not consistent with each other.

It isn’t saying “I don’t know,” it is saying “It can’t be known.” Which is actually an answer and something else entirely.

Reply on news.ycombinator.com