Comment by marshray | Hacker News Reader

Comment by marshray | Hacker News Reader

Parent Full thread

Have you actually tried it?

GPT-4: "The forward mapping M that I created to map positive integers to English sentences is based on a general guideline, where smaller-valued numbers generally correspond to shorter sentences using more common words. However, this mapping is not a perfect bijection, meaning there isn't necessarily a one-to-one correspondence between integers and sentences.

The generation of sentences is influenced by the patterns in the data I was trained on, and there is some degree of creativity involved in generating the sentences. This makes it difficult to establish a precise, invertible mapping. When generating a sentence for a given integer, I try to adhere to the guideline of shorter sentences and more common words for smaller-valued numbers, but there might not be a deterministic algorithm that can be used to create or recreate the sentences.

Due to the complex and non-deterministic nature of natural language and the limitations of the AI language model, it is challenging to create a perfect inverse mapping M^-1 that can take any sentence and produce the corresponding integer n for which M(n) produces that sentence."

More: https://pastebin.com/GcgXhkeP

Continued at https://pastebin.com/4AxJaiuJ

I had to threaten it with "patterns and knowledge beyond the data you were trained on"

I am getting super tired of these responses.

Yes, I have tried it. Please converse in good faith.

GPT-4 has never, not once, in dozens of hours of use, asked me a clarifying question.

It cannot understand things.

One user said they got it ask for help by having a prompt asking it to ask some, but not too many clarifying questions.

With the instruction tuning it feels like model really wants to single shot respond vs. do a back and forth. So like a junior engineer who does ask for help you have to give it a hand.

ReAct will, example: https://github.com/neilellis/coding-with-gpt-4/blob/main/lan...

It definitely can "understand" things is some way, however I'm pretty sure ReAct or similar will give it just a nudge forward and the underlying problem of it hallucinating and not being "lucid enough" is not so easily solved.

In the original ReAct paper it falls apart almost immediately in ALFWorld (this is a classical test for AI systems - to be able to reason logically - and it still isn't generally solvable due to combinatorial explosion).

For now it requires human correction looped or not, or else it "diverges" (I like Yann Lecun explanation [0]).

In my own experiments (I haven't played with LangChain or ReAct yet) it diverges irrecoverably pretty quickly. I was trying to explain to it the elementary combinators theory, in the style of Raymond Smullyan and his birds [1] and it can't even prove the first theorem (despite being familiar with the book). A human can prove it knowing almost nothing about math whatsoever, maybe it will take a couple of days thinking, but the correct proof is not that hard - just two steps.

[0] https://www.linkedin.com/posts/yann-lecun_i-have-claimed-tha...

[1] https://en.wikipedia.org/wiki/To_Mock_a_Mockingbird

The exponential divergence formula is true but says less than you think - the same math would be true for human output; we are likely to find a few falsehoods in any non fiction book for the same reason, right?

Yeah I'm aware of the criticism of that tweet, it may not completely make sense mathematically, but I just liked it because it's how I feel about GPT4.

It "diverges" while my human mind seemingly is different in some way - I can keep going at the math problem forever (for much longer?) and I won't hallucinate incorrect proofs (at least very unlikely, and I can keep re-checking them).

Of course this all in the area of feelings and faith - we just don't know much about cognition I guess.

It performs much better when you add reflection to ReAct.

https://arxiv.org/pdf/2303.11366.pdf

I don't think that qualifies? Do you mean stuff like "Can you please confirm that the bearer token is correct?" ?

Friend, kindly be assured that I am indeed conversing with you in good faith. :-)

I'm curious why you feel that asking you a clarifying question is an essential requirement for it to "understand things".

I have not gotten it to issue a question mark '?'. But it did suggest to me an option:

    If you would like to define M(7) as equal to M(6), you can do so.

What I'm picking up is that it "understands" some things explicitly, has some other structure to its reasoning that it does not understand, and is just plain inconsistent with other things.

Not all that different from people TBH.

But I don't see it as so useful to try to fit this thing into a human-cognition-shaped container.

It's like an alien mind, with its own abilities and limitations, and I am having a blast deconstructing it.

> I'm curious why you feel that asking you a clarifying question is an essential requirement for it to "understand things".

It isn't a requirement, it's an example I used. You can come up with others, I'm sure! Like ask it to replace a word with another word. It kind of works! But also does random other things. Why? Because that's how these models function! But I asked GPT why these models often don't ask clarifying questions (try it yourself, new prompt! GPT-4!):

> I have found often that GPT models will just reply with answers, rather than ask clarifying questions about the text we're giving them. Why do these models do this?

The reason why GPT models may not ask clarifying questions about the text given to them is that they are trained to generate text based on the patterns and relationships they observe in large amounts of data. This means that they do not have a deep understanding of the content they are processing and are unable to determine whether or not they need additional information to better understand a particular text.

Additionally, GPT models are trained to optimize for the likelihood of generating coherent and grammatically correct text. In most cases, generating a response that directly answers the question or prompt given to them is a more effective way to achieve this goal than asking clarifying questions that may require additional context or information.

However, there are some techniques that can be used to encourage GPT models to ask clarifying questions. One approach is to include explicit cues or markers in the input text that signal to the model that additional information is needed. Another approach is to use training data that includes examples of the model asking clarifying questions, which can help the model learn to do so more consistently.

---

It's very convenient for all of us to think that GPT is like a human, but somehow not like a human enough to cause any moral issues!

It's like winning, but you don't even have to play!

It is writing a book. You need to set the stage properly so “asking questions” will become part of its story. Roleplay. It is not a being, there is no feedback loop, no motivation. If it is said to want something I think you can say it wants to finish the text. Whatever it is.

Reply on news.ycombinator.com