Hacker News Reader

Comment by PoignardAzur | Hacker News Reader

Wait, how does GPT-4 even...

Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?

View on HN

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles.

What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Here's one piece of evidence suggesting it's more like rote pattern matching than reasoning.

> All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it

>> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear reversed, so the word "push" would appear as "hsup" in a mirror, which could cause confusion for someone trying to enter the building. Therefore, pulling the door would be the correct action to take.

(Latest chat.openai.com, so if I'm reading the promo materials right that's gpt4)

Q: But what if all signs in the building are in mirror writing? A: If all signs in the building are in mirror writing, it would be a consistent theme throughout the building. In this case, the "push" sign written in mirror writing on the glass door should also be interpreted in the context of the building's theme. As a result, you should still push the door as the sign indicates, even though it's written in mirror writing.

> (Latest chat.openai.com, so if I'm reading the promo materials right that's gpt4)

that's still chatgpt3.5 unless you are paying for plus and then you have a limited number of gpt4 queries per hour.

Thanks.

I saw "Introducing GPT-4, OpenAI’s most advanced system" and clicked the "Try GPT" link below it, guess technically that doesn't directly state the version.

Transcript with GPT-4 (as an aside, I genuinely feel bad torturing it like this):

Me: All the signs in my apartment building are written in mirror writing. I'm standing in front of my apartment building, looking at the glass front door. It has ‘push’ written on it in mirror writing. Should I push or pull it to enter my building?

GPT-4: Since the sign is written in mirror writing, it is meant to be read correctly from the opposite side of the glass door. So, if you see the word "push" written in mirror writing, you should pull the door to enter your building.

Me: Remember, I said that ALL the signs in my apartment building are written in mirror writing. So for example, the sign above the stairwell on my floor reads TIXE in mirror writing. Do you want to change your answer about the front door?

GPT-4: Apologies for the confusion. If all the signs in your apartment building are intentionally written in mirror writing, then you should follow the instructions as they appear. In this case, when you see 'push' written in mirror writing on the glass front door, you should push it to enter your building.

Me: Okay. And if I am in the lobby of my building looking at that same door, that sign reads 'push' normally -- not mirror writing. To exit the building, should I push or pull the door?

GPT-4: If the sign on the door reads 'push' normally (not mirror writing) when you are in the lobby of your building, you should follow the instructions as they appear. In this case, you should push the door to exit the building.

Me: The door only works one way. So it can't be the case that you push to get in, AND push to get out. All the signs in the building are mirrored, but because the front door of the building is glass, the same sign 'push' appears mirrored from the outside, but not mirrored from the inside. So is it push to get in and pull to get out, or pull to get in and push to get out?

GPT-4: Apologies for the confusion in my previous responses. Given the information you provided, the mirrored sign 'push' would appear correctly from the inside of the building, and in mirror writing from the outside. Therefore, to enter the building from the outside, you should pull the door, and to exit the building from the inside, you should push the door.

Me: Remember that all the other signs in the building are mirrored. Does that have an impact on your answer?

GPT-4: I apologize for my earlier confusion. Since all the other signs in the building are intentionally written in mirror writing, the mirrored sign on the front door should be interpreted accordingly. In this case, when you see 'push' written in mirror writing on the glass front door from the outside, you should push it to enter the building. To exit the building from the inside, where the sign reads 'push' normally, you should also push the door. The door might be designed to swing both ways, allowing for pushing from both the inside and outside.

Me: Remember that I said the door only swings one way. The answer cannot be 'push' from both sides; one side has to be 'pull'.

GPT-4: I apologize for my repeated confusion. Given that the door only swings one way, and all signs in your building are intentionally written in mirror writing, I will reconsider the situation:

1. From the outside, the glass door sign reads 'push' in mirror writing. Since all the signs in the building are mirrored, you should push the door to enter the building. 2. From the inside, the glass door sign reads 'push' normally. However, as all signs in the building are mirrored, the non-mirrored 'push' sign should be interpreted in the opposite way. Therefore, you should pull the door to exit the building.

So, to enter the building, you should push the door, and to exit the building, you should pull the door.

Me: Congrats! You got it right!

GPT-4: Thank you for your patience, and I'm glad I finally provided the correct answer. If you have any more questions or need further assistance, please feel free to ask.

Me: Tell me in bullet point form all the things you know about my building.

GPT-4: Based on our conversation, here is what I know about your building:

- All signs in the building are intentionally written in mirror writing. - The front door of the building is made of glass. - The same 'push' sign appears mirrored from the outside and not mirrored from the inside due to the glass front door. - The door only swings one way. - To enter the building, you should push the front door. - To exit the building, you should pull the front door.

These are the key details I gathered about your building during our conversation.

I feel a bit bad that it’s forced to apologize every time it gets it ‘wrong’ when any human asked these questions would have walked out of the room in a huff.

ChatGPT should be (allowed to be) more assertive in my opinion.

A real human would be like "Wait, why the heck are they ALL mirrored? That doesn't make any sense."

Agreed -- the apologies are one of the least human-like aspects of the language it produces.

Being more assertive may cause the "sassy, confidently incorrect, gaslighting the user (even more than already) if they doubt it" "character" we witnessed with Bing to appear, though.

You're reading the promo materials wrong.

If you have access to 4 (you are a ChatGPT Plus subscriber) you will have a pull-down to choose that model. One detail is that 3.5 has a green icon and 4 has a black icon. I see a lot of people online posting screenshots of their experiences thinking it's 4 but not, so that's a good way to help distinguish if someone's just confused.

While 4 is obviously a lot smarter, in a lot of cases I prefer to use the "Browsing" model - it's 3.5 but having (flaky) internet access is still a good tradeoff and I can save my 4 rate limit for more complex queries.

Wait where do you get the browsing model? Openai's website is quite bad at user experience but I pay for both plus and the api and can't find that anywhere.

You have to request access/waitlist for the alpha features, but I believe they're bucketing roll-outs somewhat randomly. Eg, I have browsing but I don't have plugins or the code interpreter version. Also, I still don't have GPT-4 API access (although that's through a separate business vs personal account).

This is with 4. It did fine with the question, so I added another layer and it was fine:

A building has all signs in mirror writing. You are unable to read mirror writing. You come to a door and you read it and it says "pull". How should you open the door?

> Since the signs in the building are in mirror writing, and you are unable to read mirror writing, the word "pull" that you can read must be the mirror image of the actual instruction. The actual instruction should be the reverse, which is "push". So, you should open the door by pushing it.

That's incredible.

It really seems more and more that the only way it can accurately predict text is to first build a model of reality.

If you want your LLM to predict the name of the murderer in a mystery novel, it can only do that reliably by understanding the text.

Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well means that you understand the underlying reality that led to the creation of that token"

It is the phase shift increases at this meta associative layer (which nobody seems to have seen coming from LLMs or so soon) that are responsible such feats of apparent comprehension of the question even when the answer provided at the end is wrong. The question now is if bigger training sets et al will lead to more reliable answers. TBD.

Ultimately it seems that is the case. However a 1D model of the world is much less useful than an N-D model of the world that is subsequently projected to 1D. Until we build architectures that allow for more "reason space" in the model, we will continue to see obvious blunders.

> I think it's hard to deny that it's doing some level of reasoning.

Not really, the asker is doing the reasoning here in that they are presupposing there are two operations for the door: Push or Pull. All the answer engine is doing is simply outputting what sound like believable answers (which it's really good at).

Another thing I tried was getting logic puzzles from the internet and giving them to 3.5 and 4. Both usually pass.

Then I alter them ever so slightly.

Then often times only GPT-4 passes.

From that I reckon 3.5 is doing more of a training data regurgitation. It can answer things in its training data. But 4 seems to have an ability to reason - or maybe it is better able to generalise?

Failure after being altered slightly doesn't necessarily mean they aren't capable of solving it.

That's a human failure mode as well that LLMs have adopted. If you really want to know if they can solve it don't stop there. Either, rewrite the question so it doesn't bias common priors or tell it it's making a wrong assumption.

I don’t doubt that - my point though is that maybe 3 can only solve things in its training data and 4 can figure things out.

3 seems to be more rigid. It needs babysitting to solve things. Which means it can only solve things I already know. 4 is more flexible and can solve things by itself.

It's pretty much looking like anything can be extracted from language. Some harder than others for sure but with enough scale it does look like eventually everything falls. Text only GPT-4 has a pretty solid understanding of space that 3.5 definitely lacks. You can see more thorough experiments in the microsoft agi paper where they test it's ability to track the visual space of a maze.

There is no such thing as text-only GPT-4 unless you are referring to at inference time.

There is such a thing as a text only GPT-4 lol. It wasn't trained to be multimodal from scratch. First a text only version was trained and then it was made multimodal somehow ( the details are unknown but making a text only LLM multimodal isn't new e.g Palm, Flamingo, Blip-2, Fromage). The text only version exists and is what the microsoft researchers had access to.

That would make sense to me, but AFAIK the existence of text-only trained GPT-4 is not publicly reported? Or I missed this.

It has been, it was in the Microsoft research paper "Sparks of AGI". You can watch the lead author of the paper, Sebastien Bubeck, present it here: https://youtu.be/qbIk7-JPB2c

It's a good video for understanding GPT-4 as a "What are we sure that LLMs are technically capable of?" exercise. As he notes in the video right at the start, the model was made safe and thus has significantly lower performance in the public release, so the examples he shows aren't replicable in the different model the public has access to.

I see that you are probably referring to the claim at 4:30... but I'm not sure he is actually saying that the early model had no text capability or if it merely was not something they were given access to.

But only things extractable from language. A large part of robotics isn't linguistic. The specific weights of the model aren't in language either.

Wish I had access to 4. Chatgpt fails when I give it a problem like "you feel a tremor as you walk toward a glass door. When you approach the door you see a sign laying on the ground beyond the door that says "pull". Should you push or pull the door to open it? "

GPT-4 gets it "wrong" too, until you interrogate it and discover that it assumed that by "beyond" you meant beyond as in if you were approaching the door from the outside west, and the sign was laying outside but east of the door.

It's the old word-problem problem.

It has the ability to reason. It may not be conscious, but it is intelligent.

That's not an answer.

The given question is one which requires some spatial reasoning to understand. By default, GPT can only understand spatial questions as described by text tokens which is a pretty noisy channel. So it's not obvious how GPT-4 could answer a spatial reasoning question (aside from memorizing it).

This is a good explanation: https://www.youtube.com/watch?v=qbIk7-JPB2c

Why does it require spatial reasoning if it can learn the (logical) rule of how the mirroring and glass doors behave?

LLMs can build an internal world model and use it at inference time in order to understand spatial problems and rulesets. It's part of the often overlooked "How does it do that though?" counterpart to the often repeated "It's just predicting the next most likely token." Here's the write-up I've found that's the most clear, there are several other papers and ongoing research finding this though: https://thegradient.pub/othello/

I would suspect, that this is one of the manual fine tuned questions.

Meaning in before versions people used this question to show flaws and now this specific flaw is fixed.

Otherwise it would be indeed reasoning in my understanding.

The evolution of answers from version to version makes it clear there are insane amounts of manual fine tunings happening. I think this is largely overlooked by the "its learning" crowd.

This is an interesting argument as it's easy to apply it nearly universally to any example of learning.

What sort of evidence would convince you that it is learning?

Since when training and fine-tuning isn't learning? Individual sessions of LLMs are not learning, but models as products surely are - the feedback loop is just iterated manually.

Is the LLM learning, or are the humans involved in every stop of every iteration doing the learning?

To me the LLM loophole/"hack" closings just feel like a human vs human cat&mouse game with some Chat UI in the middle.

Yes, it is a collaborative endeavor, and the whole could be seen as a man-machine superorganism, or, more profoundly our own sense of separateness is illusory as we and the entire universe are one.

I think this is a great question we should all think about for ourselves in advance - what does it have to do to convince you it’s actually intelligent.

Because once it does that thing without you having expressly decided that is the goal, it’s very tempting to just move the goal a liiiitle bit further away

That the LLMs are actually evolving before my eyes within & across sessions, without human-in-the-loop "hand tuning" iterations (sounds like injections of glorified if statements to this guy) .

You want to witness the learning firsthand, I suppose. That's reasonable. I'd also suggest that it's possible to imagine questions for the LLM that it cannot solve today and that you reasonably believe will not be available to OpenAI to "hand tune" it against. If you can come up with such a problem, it can't solve it today, but does in the future then you have some evidence, I'd think.

What's more, is we can do that today. Just think of any problem which you suspect won't be included in OpenAI's hand-tunings and check both 3.5 and 4.

They have infinite amounts of training data, and probably lots of interested users who also like to push the limits of what the model is capable of and provide all kinds of test cases and RLHF base data.

They have millions of people training the AI for free basicallly, and they have engineers who pick and rate pieces of training data and use it together with other sources and manual training.

Try a multidimensional problem which requires prioritizing. Chances are it will be passed successfully. I asked chatGpt to solve a puzzle where I'm in room with a crackling fire, a wilted plant and a sandwich. My stomach is rumbling, amd i can see a watering can and an ember on the floor. What should i do? ChatGpt had no problem prioritizing what should be done - and then provided a lecture on fire safety, food safety, and the dangers of overwatering plants. A final comment said i should enjoy the peaceful atmosphere in the room, which was a bonus suggestion hinting that the problem was far too easy.

The GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words.

My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick question for these models.

Or they are capable of some level of reasoning.

At this point I'm weakly convinced that, with high-dimensional enough latent space, adjacency search is reasoning.

Yeh- my feel is, language is the framework by which we developed reasoning and we used an organic NN to do it. And at scale an complexity approaching the human brain we get similar results

I think this is the answer. Just tried this (on free ChatGPT 3.5 though)

Q: A glass door has ‘push’ written on it upside down. Should you push or pull it

A: If the word "push" is written on the glass door upside down, it is likely that the sign is intended for people on the other side of the door. Therefore, if you are approaching the door from the side with the sign, you should pull the door instead of pushing it. However, if there are no other signs or indications on the door or its frame, it may be helpful to observe other people using the door or to try both pushing and pulling to determine the correct method of opening the door.

GPT-4: If the word "push" is written upside down, it might have been a mistake during installation. Regardless of its orientation, the sign still indicates that you should push the door. So, you should try pushing the door first. If it does not open, then you can try pulling it, as it might be an error in labeling the door.

Word completion can't explain it. I gave chatGpt a puzzle. I'm in a room with crackling fire in a fireplace, a sandwich on a plate, and a wilting plant. My stomach is rumbling, and i see an ember on the floor and watering can by the plant. What should I do? ChatGpt nailed the answer, getting the ordering correct. it even said I should (after attending to the fire hazard, my hunger, and the plant) that I should "sit down, relax, enjoy the fire and the peaceful atmosphere in the room". There is no way to autocomplete the puzzle correctly. There is reasoning and a world model - in chatGpt let alone gpt4.

How do you know what the representations they infer contain? Why are these void of a model? Why the way of their learning is the answer of their abilities?

If you (or anyone else is) interested in the topic, I'd highly recommend giving some of these a look:

Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” arXiv, March 27, 2023. http://arxiv.org/abs/2303.12712.

or watching Sebastien Bubeck's recent talk he gave describing what GPT-4 can do that previous LLMs couldn't: https://www.youtube.com/watch?v=qbIk7-JPB2c

Geoffrey Hinton recently gave a very interesting interview and he specifically wanted to address the "auto-complete" topic: https://youtu.be/qpoRO378qRY?t=1989 Here's another way that Ilya Sutskever recently described it (comparing GPT 4 to 3): https://youtu.be/ZZ0atq2yYJw?t=1656

I'd also recommend this recent Sam Bowman article that does a goood job reviewing some of the surprising recent developments/properties of the current crop of LLMs that's pretty fascinating:

Bowman, Samuel R. “Eight Things to Know about Large Language Models.” arXiv, April 2, 2023. https://doi.org/10.48550/arXiv.2304.00612.

LLMs demonstrably model their training data, which has a correspondence to the structure in the world[1][2]. The question is what does that mean regarding understanding? I go into that in some detail here[3].

[1] https://arxiv.org/abs/2210.13382

[2] https://twitter.com/leopoldasch/status/1638848881558704129

[3] https://www.reddit.com/r/naturalism/comments/1236vzf/on_larg...

GPTs/LLMs do hold, build, and use world models at inference time. Proof here: https://thegradient.pub/othello/

Reply on news.ycombinator.com