Hacker News Reader

Comment by galaxyLogic | Hacker News Reader

> GPT can competently engage in various semantic tasks. The real reason GPT’s answers seem senseless being that truth-telling is not amongst them

GPT can (only) mimic speech it is trained on. It can sound or read like real world human speakers it is mimicking. But it can not REASON about whether what it is saying is "true" or logically consistent.

It can not reason. Intelligence requires the ability to reason, logically, right? Therefore I posit GPT is not intelligent. Therefore it can not be AI.

I haven't used GPT but I wonder what happens if you ask it to explain its reasoning to you? What happens if you ask it whether it thinks what it says is true and why it thinks so?

View on HN

It can and does actually reason about things. It is merely unpredictable how good it is at it and when it seems to gain the ability to do it.

> It can and does reason

If ever there was a citation needed...

It has probabilistic wording sequences that incidentally share characteristics with Bayesian reasoning:

https://www.lesswrong.com/posts/x7kL42bnATuaL4hrD/bayesian-r...

But are not reasoning, just probabilistic luck of the draw.

You can indeed adjust the priors by rewriting your prompt, here for instance adding context when a student saying the dog ate her homework did produce a chewed up paper, but it's you doing the reasoning and rewriting the prompt.

> when it seems to gain the ability

"Seems" is correct. It's a parlor trick, like horoscopes and cold reading, with intelligence in the interlocutor.

There’s no sensible reason in this life to read a lesswrong post, c‘mon. That’s not a reasoning system, it’s an online religion based on worshipping Bayes’ theorem.

https://metarationality.com/bayesianism-updating

The “probabilistic” is part of the fixed-function evaluation of the model though. That’s something we humans added; if it makes it not work, just don’t do that.

Nice link, I wasn't aware of Bayenism. As to why people worship neural networks "AI" I think they like it because they are lonely. It gives them hope that there could be a machine they could talk to and the machine would understand and offer comfort and advice. I think it will, to many people, if they juts believe in it. A bit like placebo which works as well. Cargo cult, religion. "AI" will be our savior they think.

But it does not seem to know when it is right or not does it? It doesn't seem to be conscious of its own reasoning or is it? And if you ask it about its reasoning it could give an answer that might or might not be true right? So what confidence can we have that it is actually doing reasoning in one form or another?

When we see a movie on the screen we see what looks like people speaking and doing things but really it's just a recording. The images on the screen or celluloid possess no intelligence even though on the screen they seem to behave intelligently.

The difference between movies and the language models is that the so called AI-bot is like a movie which has a big tree of possible plot-choices it can take depending on how the audience cheers or boos. That doesn't mean there is any process we could call "reasoning" behind the bot's answers. You know like logical deduction.

A person can explain how they reasoned some conclusion. And so can SYMBOLIC AI. But these trained language models they just mimic the observed speech, not the reasoning processes that go on inside the brains of people who are speaking. If they were we should be able to produce a trace of such reasoning, and the language models should also be able be able to describe it to us, like humans can.

You can ask it questions noone has ever asked before, that it can't possibly have memorized the answers to, and simply say "explain the answer step by step". It's not guaranteed that it will answer correctly, but it often does answer correctly and with a chain of reasons.

The cases where it answers incorrectly are sometimes reasoning failures, sometimes because it doesn't know its sources (because the model training doesn't "cite its sources" when dumping knowledge in there), and sometimes because of fixed-function parts of the model (like how it doesn't see individual numbers/letters, but rather tokens using compressed "byte pair encoding".)

You're tricked by probabilities.

In the joke explainer example posted here a couple weeks back, the "chain of reasons" why each joke was funny was dead wrong though probably right for most jokes. It has no reason, only likelihoods.

Your belief it has a chain of reasons is usually confirmed because the dumb probability-selected list is, by design, probably plausible.

> It's not guaranteed that it will answer correctly,

Why? Because its reasoning-algorithm is not correct, because it has none. All it has is statistics, likelihoods, which makes it look like as if there was some reasoning behind it.

Statistics are the only thing anything has. You don’t have an algorithm either.

Or rather, you (or a CPU or an ML model) have a “statistical” base layer and then other abstractions on top of that which are capable of being (relatively) deterministic on top of the unreliable execution machinery.

> You don’t have an algorithm either.

Are you saying algorithms don't exist, can't be followed and reasoned about, and proven correct, or not?

Why do you offer an opinion without spending time any time with the thing you are talking about?

Why? Because opinion is already half-intelligence :-)

Big assumption that anyone’s reasoning carries any weight if a chat bot trained on billions of our interactions shows no signs of doing it

Reply on news.ycombinator.com