Comment by Version467 | Hacker News Reader

Comment by Version467 | Hacker News Reader

Parent Full thread

He is far from the only one.

If you're interested in exploring this further I can really recommend taking a look at some of the papers that explore GPT-4's capabilities. Most prominent among them are the "Sparks of AGI" paper from Microsoft, as well as the technical report from openai. Both of them are obviously to be taken with a grain of salt, but they serve as a pretty good jumping off point.

There are some pretty good Videos on Youtube exploring these papers if you don't want to read them yourself.

Also take a look at the stuff that Rob Miles has published over on Computerphile, as well as his own channel. He's an Alignment Researcher with a knack for explaining. He covers not just the theoretical dangers, but also real examples of misaligned ai, that alignment researchers have predicted would occur as capabilities grow.

Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI. It's still unclear that it will get us all the way there, but recent developments have made a lot of ai researchers rethink that possibility, with many of them significantly shortening their own estimates as to when and how we will get there. It's very unusual that the people that are better informed and closer to the research are more worried than the rest of the world and it's worth keeping this in mind as you explore the topic.

I read that pre-print Microsoft paper. Despite the title, it doesn't actually show any real "sparks" of AGI (in the sense of something that could eventually pass a rigorous Turing test). What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness; our brains seem to be wired that way and this is likely the source of most superstition.

https://arxiv.org/abs/2303.12712

While there is no scientific evidence that LLMs can reach AGI, they will still be practically useful for many other tasks. A human mind paired with an LLM is a powerful combination.

>What the paper actually shows is that even intelligent people have a bias towards perceiving patterns in randomness

I'm not saying that you're wrong, but...

you'd have to provide a more rigorous rebuttal to be taken seriously.

AGI can exist without sapience and intelligence is a continuum. you can't just hand wave away GPT's capabilities which is why the sharpest minds on the planet are poking this new machine to work out wtf is going on.

human intelligence is a black box. we judge it by its outputs from given inputs. GPT is already producing human-like outputs.

a common rebuttal is: "but it doesn't *really* think/understand/feel", to which my response is: ...and? ¯\_(ツ)_/¯ what does that even mean?

I was just demonstrating its capabilities to a client. I asked GPT 4 to summarise a cloud product in the style of Encyclopaedia Dramatica, and it came up with a unique phrase not seen on the Internet when talking about auto-scale: “It’ll take your wallet on a roller coaster ride.”

What’s brilliant about this is that typically auto scaling metrics look like a stereotypical roller coaster track with the daily ups and downs!

That’s a genuinely funny, insightful, bespoke, and stylistically correct joke.

Tell me that that is not intelligence!

There's a certain amount of cosmic irony involved whenever someone calls LLMs 'stochastic parrots' or whatever.

How do you know this was unique and not picked up in its training set?

I searched Google for a few variations and turned up nothing.

Agreed.

Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making.

Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical?

Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.

>Could you really be confident you understood how it worked by just running experiments on it as a black box

the human brain is a black box, we can certainly learn a lot about it by prodding and poking it.

>Things look like magic especially when you can’t look under the hood.

imagine we had a 100% complete understanding of the mechanical/chemical/electrical functioning of the human brain. Would knowing the magic make it any less magical? in some sense, yes (the mystique would be gone, bye bye dualism), but in a practical sense, not really. It's still an astonishingly useful piece of grey matter.

I don't think static LLMs could reach AGI tbh. An LLM is like slicing out the language processing portion of our brain.

Well realistically it's like independently evolving the language processing part of our brain without forming the rest of the brain, there seems to be extra logic/functions that emerge within LLMs to handle these restrictions.

I think we'll see AGI when we finally try to build one up from various specialised subcomponents of a "brain". Of course GPT can't "think", it only knows how to complete a stream of text and has figured out internal hacks during training to pass the tests they set for it.

The real difference will be when we train a model to have continuous, connected abstract thoughts - an LLM can be used to communicate these thoughts or put them into words but it should not be used to generate them in the first place...

> Also I think it's important to mention that just a short while ago virtually no-one thought that shoving more layers into an llm would be enough to reach AGI.

This was basically the strategy of the OpenAI team if I understand them correctly. Most researchers in the field looked down on LLMs and it was a big surprise when they turned out to perform so well. It also seems to be the reason the big players are playing catch up right now.

I think it was a surprise the behaviors that were unlocked at different perplexity levels, but I don't really agree that LLMs were "looked down on."

Maybe not "looked down on", but more of "looked at as a promising avenue". I mean, 2-3 years ago, it felt LLMs are going to be nice storytellers at best. These days, we're wondering just how much of the overall process of "understanding" and "reasoning" can be reduced to adjacency search in sufficiently absurdly high-dimensional vector space.

People certainly knew that language modeling was a key unsupervised objective to unlock inference on language.

I agree that I think they underestimated quite how useful a product could be built around just the language modeling objective, but it's still been critical for most NLP advances of the last ~6+ years.

Reply on news.ycombinator.com