Hacker News Reader

Comment by yosito | Hacker News Reader

Yeah, I gave it a simple task of encoding a secret message in a sentence by using the first letter of every word. Hello = "Hey everyone, lick less onions". I worked with the prompts for over an hour to try to get it to complete the task, and while I did have some success, it really struggled to reason about the task or provide a valid response. If it can't even reason about a child's game, I can imagine it struggles with a programming language it has barely seen. I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.

View on HN

> I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.

It turns out that humanity’s problem might not be that AIs can think but rather that humans believe that AIs can think. One might even go so far as to say there’s a real danger that we hallucinate that AIs can think, to our detriment.

We don’t actually know what “thinking” is, though, so I’m not sure it’s possible to say “this model can’t think”.

> We don’t actually know what “thinking” is

How about: thinking is to make sense of the world (in general) and decide how to respond to it.

Ai definitely senses and definitely makes decisions. It does not feel. But it understands concepts. Just like people don’t understand everything—and you can test them to see what they understand—AI understanding can also be assessed with benchmarks. If we don’t base AI understanding on benchmarks, then we don’t really have a grounding.

Do we “really feel?” Or is that just our subjective interpretation of our goals? (At the risk of falling into a no true Scotsman argument)

> Ai definitely senses and definitely makes decisions.

To sense is not the same as to make sense.

Now you’ve just rephrased thinking. What is „making sense of the world“?

It seems one of the core components of human-level thinking is the ability move beyond just a recomposition of what you already know. Not long ago the epitome of expressible human knowledge was *emotive grunting noise.* Somehow we went from that to the greatest works of art, putting a man on the moon, and delving into the secrets of the atom. And we did it all exceptionally quickly once you consider how little time was spent dedicated to advancement and our countless behaviors that tend to imperil, if not reverse, advances.

> problem [...] that humans believe that AIs can think

Definitely some people are going to believe this, eventually?

People already bow to statues and worship various invisible gods -- the AI programs would be so much simpler to start worshiping? When they can speak (if there's a loudspeaker) and generate text about them being alive and don't want to be switched of. And that everyone should obey their command. -- Wait 15 years and we'll see what new sects have appeared?

You will not need 15 years - I'd give it the next election. Someone just needs to turn Q into a chatbot and we're basically there.

It’s like Twitter, but with bots.

I am personally more worried by the concept that potentially humans believe that humans can think, and in reality, what we consider to be intelligence is not much more than a flesh-and-bones LLM.

You probably meant it in tongue and cheek way (I can’t tell) but I think a lot of our fear / hesitation / denial about how useful these models are is buried into this idea that perhaps we are not special and not fundamentally different than these models.

My tone was flippant but I did mean what I said. I agree with you on this.

This is probably a tokenisation problem: https://news.ycombinator.com/item?id=33915287 (which is an artefact of the way the model was trained)

Bing in creative mode really struggled with this task, and I don't think I could have successfully forced it.

Bing in precise mode had no trouble at all.

Me - Let's generate an acrostic sentence for "jack and jill went up the hill"

Bing - Sure! Here’s an acrostic sentence for the phrase “jack and jill went up the hill”:

Just as children know and never doubt, joyfully in love, laughing while enjoying nature together under pure trees, having endless happy interactions, laughing loudly.

A truly "intelligent" AI would probably have told me that the name for this is an acrostic sentence.

Now that I know, I tried a few more prompts.

Me: "Can you generate an acrostic sentence for "hello"?"

GPT-4: "Happy elephants love leaping over obstacles."

Me: "Can you pick a secret word and generate an acrostic sentence for it?"

GPT: "Secret word: KANGAROO. Kindly assisting neighbors, generous animals roam openly outdoors."

Me: "Can you pick a secret word, and without telling me the word, generate an acrostic sentence for it?"

GPT: "Playful, endearing creatures happily yawn, turning every day into a lovely event."

Me: "What was the secret word?"

GPT: "PECHYTIALE"

It's interesting that GPT seems to need to write the word first before making an acrostic sentence for it. Seems to me like a perfect illustration of the fact that it's just generating likely responses one token at a time rather than having any awareness or thought.

Another evidence is to ask GPT for a link with a reference for the answers it gives: it'll generate them instead of copying them.

I'm not sure, but I got it to work great on my first try with the following prompt:

------ Your task is to encode a secret word by making a sentence where each word starts with a letter in the secret word in order. For example, encoding the secret word 'bag' could produce the sentence 'bagels are green'. Encode the following secret word: 'pen' ------ People eat noodles. ------

Worked for window, backpack as well, although I did have to tell it not to use the secret word in its encoding when I got to backpack, and then to follow the same order and not repeat words after a few attempts.

> I don’t think it’s actually reasoning about things at all

This is a huge leap. There’s plenty of humans who couldn’t do that, especially historically.

Stop thinking about reasoning as a binary question and think of it as a spectrum. Actually, not even a spectrum, but a huge multi-dimensional problem. ChatGPT does better at some reasoning problems than most humans do, and worse at some others. You clearly found one that it’s particularly bad at.

I think the most interesting response I've gotten was one where gpt-4 noticed halfway through a response that it had made an error, apologized and then put the corrected sentence. When I queried it, it claimed it generates the response a token at a time, and could not back up when it realized the message was incorrect, but I don't know enough about how the tech works to ascertain the veracity of the statement.

That's exactly right. It can see what it's returned but can't edit it.

Is this with Chatgpt or GPT4? The latter is supposed to be way, way better, according to this paper[1]

[1] - https://arxiv.org/pdf/2303.12712

ChatGPT has a GPT-4 mode if you're paying for it.

ChatGPT doesn't understand the components of words (letters, syllables) very well.

Reply on news.ycombinator.com