Hacker News Reader

Comment by davnicwil | Hacker News Reader

> 4 years ago, a program that could create photorealistic pictures, talk to you in any language of the world and solve the hardest math problems that we know, we would have called it AGI.

I keep seeing this idea and I don't understand the reasoning behind it.

I think it could be a bit like saying if you showed someone 500 years ago a smartphone they would likely conclude at first it was magic. But once you had some time to let them use it and tell them how it all worked on a high level they would eventually obviously realise, no, it's not magic.

I guess just in the same way if you presented current LLM tech out of nowhere a few years ago to someone who'd never seen it, I concede they may be likely to imagine it was AGI in that first conversation, depending on their background.

But after using it for a bit and learning what an LLM is etc they'd land exactly where everyone is today - a great technology useful for some things, not AGI, not magic.

View on HN

Wait, are you literally saying that modern LLMs are not "magic" (or AGI) because we know how it works?

If that is the case, then it trivially follows that humanity will never be able to build AGI, because every time we build something new, we will understand how we built it, which disqualifies it from being an AGI.

But that doesn't seem like a useful definition to me.

(Edit: I fully agree with your comment, just wanted to pick on a small point)

Thing is, we don’t understand how LLMs work. Not really. And everyone at the forefront of this space has said so. We know how to grow them but that is not anything like the same thing as understanding why or how they actually function. Our level of understanding of LLM intelligence is only slightly ahead of our understanding of the neuroscience of human intelligence, and no one would say we understand that at all.

So I think describing LLMs as magic at this point is absolutely appropriate.

I know how they work pretty well and most days I still have moments where I am struck by how bizarre and magical these thinking machines are.

The fact that a monkey can pick up an artifact a size of a small mirror and do a video chat, see and talk to another monkey on tge other side of the planet is absolutely bizarre. There's no good reason why this should be allowed.

I'm finding it more bizarre, if compared to creating a virtual talking monkey from scratch, with a little bit of Python and a large calculator.

What ? If that experiment was done on monkeys I don't think they had to use Android / iOS / iPadOS interface themselves. All they had to do is push a button

we are the monkeys

Hey, I've never said that, it is allowed with actual monkeys.

But I didn't talk about magic but AGI. Or you think AGI needs some kind of mystical element?

What would convince you that it is AGI?

I always here things like "oh it's useful but dumb on some things", but it's just vague.

What is the test? What is a question that it fails at compared to humans? And no, you can't just say "find me the cure for cancer", but I believe there is probably enough intelligence in the weights that there is likely a cure in there with enough compute and the right questions.

The fact that you need to ask "the right questions" is why it's not AGI. A general intelligence should be able to ask of its own volition the interesting questions required to advance its goals.

Most humans can't and don't ask these questions.

we're not talking about most humans here, we're talking about the capabilites of the universe of humans.

Why?

Humans don't ask the right questions otherwise they could get the answers w enough compute. So humans aren't agi?

Again w the vagueness. It's not agi because it doesn't ask the right questions. What are the right questions? Who knows.

You're proving my point

AGI doesn't mean Omniscience

I can’t predict how a novel intelligence could prove to me that it is intelligent. A novel intelligence would have to work out how to do that for itself

i think we'll know agi when we see it, but we can't really predict what that will look like

When it can replace a white collar remote worker without humans team realizing they work with the machine. And no, Agents are not like this whatever initial prompt and set of skills you give them. They still won't progress, learn and apply that knowledge like a human.

sounds like the Turing Test all over again

> What is the test? What is a question that it fails at compared to humans?

Human intelligence does not consist of answering questions.

And every day I have to say "No, you stupid clanker ...", so if this is "general intelligence", it's the intelligence of an idiot savant who is good at calculations but sucks at most other forms of intelligence.

Reply on news.ycombinator.com