Simplest thing GPT4 can’t do? | Hacker News Reader

Simplest thing GPT4 can’t do?

lesswrong.com

6 pointssashank_15099 comments

I’ve found it quite bad at unscrambling letters. Asking a seven letter word from “solution” I got , outlins (not a word), loutins( a long explanation about how this relates to hawks which was also made up, this is not a word) and after lots of prodding I managed to get it to say lotions

The fact that it eventually got "lotions" is pretty surprising and impressive, honestly. Can you share how you "prodded" it?

First I asked it, what is something you rub one your body. It answered sunscreen. Then I asked can you make a 6 letter word from given letters. It answered lotion. Then I asked can you make a 7 letter word and it just repeated its previous answer that no 7 letter English word is possible. Finally I told if you extend the 6 letter word you came up with, it should be possible and then it apologized and came up with lotions

That's not mistakable for anything close to understanding language, but it's still amazing

Performing this would require a lot of compute resources.

Perhaps any tasks that require a great deal of dynamic memory/comparison would be expected to struggle in this architecture.

Since "write down all possible permutations of these letters" is like n! "cross reference the permutations with a dictionary/definition if you feel like it" must be performed m*n times, maybe other such n! tasks are doomed until more parameters are feasible.

It almost certainly could produce a bash script to do that based on what I've seen.

Wordle. Even though it can recite the rules it can’t stop putting yellows in the same position. Try PEACH as the first guess where ACH are yellow then ask it to make the second guess.

I have some success if in each subsequent prompt I include a reminder about which letters ChatGPT has guessed, and accumulate and present everything a bit like a game of hangman.

That should be prompted, but it's lazy about sorting through it's own outputs like this.

Say you explain the rules and it guesses, "pecks",

> P E _ _ _

> The word contains a C

> The word doesn't contain K, S

Re-promoting and accumulating those details like this, ChatGPT 3.5 managed to solve yesterday's wordle successfully, but it took 10 tries to land on this prompting method.

Maybe more parallel GPTs instructed like, "find 5-10 key facets of reasoning about the problem presented", and then "extract those facets from the conversation, organize them into a table, and use them to supplement decision-making" might help prevent the needed parameters from blowing up too much.

This tallies with the way I’ve seen someone play chess with it. I tried and it kept making invalid moves. The prompt approach that worked was presenting all moves to date each time. So it has a very short to no recall for some things.

I mean, from the structure of things that makes sense.

If you think of it as a statistically-derived processor/algorithm, unless there's some great task of short-term memory/memorization it's forced to undergo it's going to be biased towards more direct responses.

Another thing to consider, regarding the fine-tuning, is that "instruction" may be biased towards sentence ordering, so that the actual order with which you present your prompt can be important.

For example Prompt: <detailed situational specification>. <task to be performed> might perform considerably worse than <task to be performed>. <detailed situational specification>.

I suspect it's just a short-mid-term resource constraint though since:

- demonstrating extremely sharp memory performance conversationally isn't that common

- total bout / token width isn't particularly wide in training

- presumably fewer "transistors" than humans -- basically even 200B parameters isn't all that many

Another concern that I'm not sure how to consider is that 4-letter tokenization could be an obvious reason ChatGPT is such a poor speller.

Reply on news.ycombinator.com