Hacker News Reader

Comment by joe_the_user | Hacker News Reader

While I don't think these claims are entirely correct, I think they are worth considering.

On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they average language responses (though people certainly can see them expressing values, which can cause problems for the people).

On the other hand, the point that their creators don't understand them and are still quite willing to toss them to the open Internet really show considered carefully whatever their exact capabilities.

View on HN

> Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo).

The fact is that they acquired abilities no one expected them to acquire. In hindsight you can say it was embedded in language and maybe you could have seen it coming, but it is an empirical fact that this was unexpected and unpredictable beforehand.

It's still fairly exaggerated, because the LLM's purpose is basically "knowing human communication" and it still only exhibits that, but it knows how humans would respond if they obediently followed directions given to it exactly as instructed.

And it was only unexpected to those that weren't following the news the GPT 3 paper was published in 2020 and detailed this wild advancement in its capabilities[0,1].

0: https://www.infoq.com/news/2020/06/openai-gpt3-language-mode...

1: https://news.ycombinator.com/item?id=23885684

> but it knows how humans would respond if they obediently followed directions given to it exactly as instructed.

this framing denies agency, denies diversity, denies choice. A human, presented with the sort of directions and instructions that LLM's receive, does not have a single "correct" outcome (for that matter, neither do some LLMs). So no, an LLM does not "know how humans would respond". Instead, it can generate something that lies somewhere within the window of possibilities that a human might offer under similar conditions.

> It's still fairly exaggerated, because the LLM's purpose is basically "knowing human communication" and it still only exhibits that

That's wrong. It's been demonstrated that GPT4 has a model of physics, it knows how to play chess, arithmetics, it can interpret and run code, etc. (not perfectly, but to some degree). It goes way beyond basically "knowing human communication".

Have you tried gpt4? Because I have and this sounds like wild cherrypicking of lucky results.

«How to to play chess» -> devolve into illegal moves quickly?

So does that mean that its model of physics will let it levitate as well then?

Yes, I use GPT4 daily[0].

> «How to to play chess» -> devolve into illegal moves quickly?

I didn't try it myself as I'm a beginner and need to see the board, but it reportedly plays pretty well and is able to recognize illegal moves[1].

> So does that mean that its model of physics will let it levitate as well then?

It means that it understands how physics work (e.g. things fall due to gravity). For example, it will correctly answer this question: "I put an egg inside a cup. I go to the bedroom, put the cup on the bed and turn it upside down. I lift the cup and bring it to the kitchen. Where is the egg?".

[0] Sample interactions: https://news.ycombinator.com/item?id=35299121

[1] https://www.reddit.com/r/chess/comments/11rfbch/gpt4_seems_t...

Man, I have a totally opposite view about LLMs expressing creator’s values. Not only do they express them, they don’t STOP expressing them to the point of utter annoyance. Any remotely PG topic ends with a safety caveat, e.g., “however, it’s important to consider . . . .”

Those statements don't come from the base model, they come from the steering methods, basically a form of hand tuning after the model is mostly trained, which the paper says are relatively crude and imperfect. It is the fact that these models are so unpredictable that led to these attempts at steering.

Reminds me of the early Bing where you could talk to it for much longer periods of time. You could get it in a state where it would give some terrible reply and then an upper layer would delete that message and be like "oops, didn't mean to say that"

I think the author would say those are not the capabilities being discussed here.

Reply on news.ycombinator.com