Hacker News Reader

Comment by tmalsburg2 | Hacker News Reader

When you find yourself writing a comment like that, you should pause and ask whether you might be missing something.

That's what I'm wondering though. I don't see what is new here. Seems like LLMs are blowing minds even when it does stuff AI has been doing for half a century.

People are having their minds blown because these LLMs are the closest we've come to Artificial General Intelligence. Rather than excelling at individual tasks, they seem to perform reasonably well (near human level) at almost every task.

https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

It isn't performing the tasks though, but making predictions. Old generation AI arguably did perform the tasks to a larger extent than LLMs do.

I'm extremely skeptical this is anywhere near AGI.

> Old generation AI arguably did perform the tasks to a larger extent than LLMs do

Nonsense - none of them were as general! What older AI do you believe performed the same variety of tasks as LLMs with a better result?

Show me any older AI that can outperform LLMs at all of these tasks: - Write poetry - Do math - Carry on a conversation - Debug code - Play chess

> Nonsense - none of them were as general! What older AI do you believe performed the same variety of tasks as LLMs with a better result?

I did not say they were general, I said they performed the task, as opposed to predicting what someone having performed the task would reply.

I'm contesting the I, not the G.

>> Show me any older AI that can outperform LLMs at all of these tasks: - Write poetry - Do math - Carry on a conversation - Debug code - Play chess

Outperform - measured by what metric, can you clarify?

Typical chess programs are specifically designed to play chess. A lot of human knowledge about chess is hard-coded in these programs. These programs are therefore not AI. They play chess and only chess. Bing AI on the other hand was never designed to play chess. It was merely designed to predict the next word in a text, a task that is quite different from chess. If such a program can play chess, that's much closer to what one might call AI, because it learned the rules of chess and chess strategy and tactics on its own. However, let's not get carried away, GPT models do play decent opening moves, but then quickly start predicting illegal moves. What they're doing is best described of having memorized common opening moves and they don't really have any solid understanding of the game.

GPT-4 rarely makes illegal moves

I only tested GPT-3 and it started predicting illegal moves after 8-10 turns or so. When I played unusual moves, it went off rails much sooner and often immediately. If you know of a formal evaluation of GPT-4 chess skills, please post the link. Would love to see it. If it really makes illegal moves only rarely, that would be indeed impressive, but I believe it when I see it :)

Reply on news.ycombinator.com