Hacker News Reader

Comment by cm2187 | Hacker News Reader

My understanding is that it basically interpolates between the most frequent sequences of words, based on context. There is no concept of expressing a rational reasoning underpinning the words. I don't think it is comparable at this stage. It is just something very good at throwing you back segments of text it has found on the internet.

View on HN

What you are probably missing is that for the model to be competent at predicting the next the word IT HAS to learn how to reason as a human being would.

If it quacks like a duck it must reason like a duck? Here I am quacking alone in my office, I guess I know how a duck thinks.

I don't see why this is. Humans aren't trained to perform a statistical analysis of what the next word is likely to be. We learn language from other humans in social and environmental context. The LLM is just being trained on text.

Nobody is saying these models are trained the same way humans are. I am just saying that during their training task (predicting the next word) they end up developing human reasoning. That of course wasn't obvious before these large models appeared. But now it's pretty clear that's the case. It wouldn't be able to perform so well in so many tasks if it didn't have something resembling reasoning capabilities.

I asked ChatGPT about the Chinese Room and asked for references. The first it came up with was "Minds, Brains, and Programs" by John Searle (1980).

The next two were, "The Chinese Room: Searle and Artificial Intelligence" edited by John Preston and Mark Bishop (2002) (Preston and Bishop did publish Views into the Chinese Room in 2002, but I can't find anything by the given title) and "The Chinese Room Argument" by David Chalmers (2006) (Chalmers has written on the topic, and published three articles in 2006, that's not one of them.

Also:

"Two cities: one, a city of “knights”, and the other of “knaves”. The residents of the two cities like to visit each other. One day, a man enters one of the cities, but doesn’t know which. He decides to ask the first person he meets where he is. The answer he receives is that he is in the “knaves” city."

produced the result:

"The man cannot be sure that the person he met was telling the truth, because the person could be a knight from the city of knights, who always lie, or a knave from the city of knaves, who always tell the truth.

"To determine which city he is in, the man could ask the second person he meets the same question. If the first person was a knight, then the second person would also be a knight, and would lie about being in the city of knights. If the first person was a knave, then the second person would also be a knave, and would tell the truth about being in the city of knaves.

"By asking the same question to multiple people and analyzing their responses, the man can eventually determine which city he is in. This problem, known as the "Knights and Knaves" puzzle, is a classic example of logical reasoning and problem-solving."

It seems to recognize the "knights and knaves" form of puzzle, but doesn't get the knights and knaves right. And it's not getting the reasoning part at all even given varying prompts.

I don't understand what point you are making. Are you saying that it lacks any reasoning capability because it fails this test? I say try a bunch more tests and then get the average to really have a better sense; one test is meaningless.

How many opportunities would you give someone who, when asked to justify their answer, routinely made up references, and who ignored important parts of a question and whose answer wouldn't work anyway?

As it turns out, we kinda do know what is going on in a LLM, and there's no model of reality for it to base reasoning on in there.

Does that mean that's impossible? No. Does it mean seeing reasoning there is making the same mistake as those who enjoyed chatting with Parry and Eliza? Probably.

I hear this type of opinion expressed frequently by coders. I use ChatGPT pretty much every day (to help with various tasks) and there's just no way to hold on to this kind of belief after you get to "know" it at that level. It clearly can reason, maybe not in exactly the same way as a person. It is superior to the average person just in terms of following a logical argument in many cases. Maybe that is a low bar but I can't help but think that it is much higher than you think.

Try asking it one of Ray Smullyan's logic puzzles, like "There are two cities: one, a city of “knights” who always tell the truth, and the other of “knaves” who always lie. The residents of the two cities like to visit each other. One day, a man enters one of the cities, but doesn’t know which. What question should he ask to find out?"

I had to add the part about the behavior of knights and knaves, because ChatGPT seems to recognize the puzzle but gets them wrong, and it seems to be completely ignoring the visiting behavior. Further, the answer doesn't really work.

Interpolation is a leaky analogy. With scale these models acquire increasingly sparse discrete-like activation patterns, and there are techniques[1] amounting to pinpointing the exact locations of the knowledge and updating it freely.

It seems that nonlinear interpolation in a rich enough latent space can give you a sufficiently good approximation to reasoning.

1. https://arxiv.org/abs/2210.07229

Reply on news.ycombinator.com