Comment by Closi | Hacker News Reader

Comment by Closi | Hacker News Reader

Parent Full thread

> It can only do what it was trained to do: make a good guess at what word should come next.

Disagree - GPT's mechanics are that it guesses what word/token should come next, however it is now displaying emergent capabilities which do show an agent which can generalize. Maybe not to 'AGI' levels, but it is starting to show little signs of it.

> It can't do basic maths, or even follow instructions. It can't make valid moves in a game.

It can do basic maths - better than most humans (but it cannot do maths better than a calculator). If I ask GPT4 to calculate 51 times 102 it will get the answer, despite most humans struggling with this task. It does this just via internal representation without any specific 'calculator' API call or functionality.

It can follow instructions - if I tell it to take the sentence "abcd" and then tell it to follow the following steps: "1) swap b and c around, 2) reverse the string, 3) ignore step 1", it will output "dcba" and describe how it reached that output

It can make valid game moves - Without being formally taught chess it will make valid chess moves most of the time (see: https://www.lesswrong.com/posts/xyjhFCSSXZsW6HDBb/a-chess-ga...). It made 29 valid moves before making invalid moves. Again, GPT4 has not been specifically trained to play chess, it just has picked up some chess rules and games while reading the internet and this has somehow formed some sort of internal representations for chess within it's model. I can also describe a new/novel game or puzzle to GPT4 and it can have a go at playing and solving it.

What it can't do is learn at the moment, although learning isn't part of a lot of definitions of intelligence, which is why we have to define intelligence before you can say if it is/isn't intelligent.

It doesn't even "understand" words and what they are made of. This is an extremely simple task that a child could do, and yet:

- Give me a list of US and European officials with names ending in "-land"

1. Mary Landrieu - former US Senator from Louisiana 2. Harold Landry - US Congressman from Tennessee 3. Jacky Rosen - US Senator from Nevada 4. Chris Van Hollen - US Senator from Maryland 5. Brian Mast - US Congressman from Florida 6. Roland Rudd - British businessman and political campaigner 7. Sadiq Khan - Mayor of London, England 8. Martin Schulz - former President of the European Parliament 9. Angela Merkel - Chancellor of Germany 10. Mark Rutte - Prime Minister of the Netherlands 11. Stefan Löfven - Prime Minister of Sweden 12. Erna Solberg - Prime Minister of Norway 13. Alexander Stubb - former Prime Minister of Finland 14. Kaja Kallas - Prime Minister of Estonia 15. Xavier Bettel - Prime Minister of Luxembourg.

We can point to a handful of strange examples where GPT4 appears to be unsuccessful, but that ignores the buckets of examples where it knocks things out of the park.

And the areas where it isn't successful appear to being closed quickly as new discoveries are found - GPT4 is dramatically better at maths and truthiness than GPT3.5 for instance (still not perfect at either, but improved enough).

GPT4 can do things a child can't do, and a child can do things that GPT4 can't do. You don't have to be able to do everything a human can do to be classed as intelligent, and GPT currently underperforms humans in some areas and overperforms them in others.

Given a list of officials, absolutely, and if there is no list, nobody would just start spouting off names that don't match the rule at all -- and the failure here is due to inability to apply basic stated directions in a logical manner. Most likely this is due to the processing of tokens not being on a character level, but it will also imply that it possesses character-level language knowledge if you ask it to produce a list of infrequent trigrams in the English language:

> As an AI language model, I do not have access to a pre-existing list of the least frequent trigrams in English. However, I can generate a list of some of the rarest trigrams based on the frequency of occurrence in a large corpus of English language text. . . . (list follows)

How can I be sure of this list it claims to generate when I have evidence it can't identify substrings?

Inability to complete certain tasks does not mean that the system is not intelligent.

It’s possible for a system to be intelligent and also fail at these questions.

IMO a system is intelligent if it can answer some questions that require intelligence (by definition) - it does not have to be able to answer all questions.

I hadn’t heard of “truthiness” outside of Boolean comparisons in programming languages, but when I looked it up, it doesn’t seem like something worth aspiring to:

> Truthiness is the belief or assertion that a particular statement is true based on the intuition or perceptions of some individual or individuals, without regard to evidence, logic, intellectual examination, or facts. Truthiness can range from ignorant assertions of falsehoods to deliberate duplicity or propaganda intended to sway opinions.

https://en.m.wikipedia.org/wiki/Truthiness

Yeah the previous comment is almost completely wrong. But on last point, it didn't get here by hard-coding and the reinforcement learning part at the end i think shows it definitely can "learn". I guess one thing I would still point to is it has a weakened sense of "self" still I think, which goes inline with it distinguishing things learned in training vs in deployment (though maybe this is just intentional idk)

Yes I agree - I really meant it doesn't directly learn from your interaction (i.e. it forgets you spoke as soon as you close the session, and only learns indirectly through the next training run - your words do not affect it's internal representation in real time like humans). The learning appears to require more information than humans too, who can do 'more with less'.

You are right that it clearly can learn though, the learnings are just baked-in with each cycle.

I think the Microsoft Tay incident is reason enough that OpenAI know it would be a bad idea to structure things in such a way that the internet can wreck it in a manner of hours, but I certainly think they could make it learn in real-time if they wanted to. They are running RLHF at least with the current ChatGPT, but in theory maybe they could do something like each user gets their own LoRA and it uses their conversations to fine-tune a version specific to them.

I suspect we'll start seeing ideas akin to that tried in the open source community.

> I can also describe a new/novel game or puzzle to GPT4 and it can have a go at playing and solving it.

It’s interesting that you brought up chess. It can do chess reasonably because there is a huge amount of chess data on the web. In that sense, it is not too surprising to me. If someone several years ago had said “I scoured the entire internet for chess-related text and fed it into an AI model, and it can play at a low-amateur level” I would be impressed but I wouldn’t be hailing a new era of general intelligence.

An example that illustrates that huge amounts of specialized data is needed for it to do any particular task: I fed GPT-4 the rules of Duck Chess. Duck Chess is exactly like regular chess but after each move, the player who just moved takes the rubber duck and places it on any empty square (there is just one duck shared between the players, and you have to move the duck: you can’t leave it on the same square for consecutive moves). Pieces cannot move through or stop on the duck. The game eliminates the concepts of check and checkmate, and ends when a player captures the opposing king.

I have given a description of duck chess to many humans (yes, I love duck chess!) who are usually much worse than ChatGPT at regular chess. When these humans play duck chess for the first time, they intuit some basic principles: use the duck to block natural developing moves for your opponent in the opening; you can often capture a defended piece without consequence by placing the duck between your capturing piece and the defender; if you want the duck to not be on a certain square for your next move, then put it on that square after your own move, since your opponent is obligated to move it; and so on.

GPT-4 meanwhile utterly fails to play the game. More often than not, it will try something illegal: putting the duck on an occupied square, passing a piece through the duck as though it weren’t there, or attempting to capture the duck after being told that’s not possible. When it does play legal moves, the duck placement is nonsensical. When asked why it placed the duck where it did, it betrays a lack of basic understanding of the rules. Its explanations tend to forget that its opponent gets to move the duck themself after their move.

This is where the “but humans make mistakes too!” arguments break down. No human who can play regular chess at the level of GPT-4 would continually struggle to make legal moves in duck chess. 99.999% of them would make better moves than GPT-4.

To me, this supports the idea that GPT-4 is great at finding and exploiting patterns that it has seen millions of times in the training set. When you veer off the training data (and your problem isn’t a trivial interpolation of related concepts that are in the training data) it seems to fall apart completely.

As one more example, GPT-4 contains some very basic facts about the game Arimaa, an abstract strategy game like chess. It can recite the rules perfectly. But I can’t play Arimaa with GPT-4 because it fails on the very first step: choosing how to arrange your pieces. I once exhausted all 25 of my messages trying to get it to make a legal configuration of its pieces to start the game, to no avail.

Reply on news.ycombinator.com