AGI is not a continuum from LLMs; true intelligence is characterized by comprehension, reasoning, and self-awareness, transcending mere data patterns.
What LLMs have done is really redefine my internal definition of "intelligence."
Putting aside the fact I don't believe in free will, I'm no longer sure my own brain is doing anything substantially different to what an LLM does now. Even with tasks like math I wonder if my brain is not really "working out" the solution but merely using probabilities based on every previous math problem I have seen or solved.
Thus AGI needs to be able to learn something new with similar amounts of data as a human, or else it isn't an AGI as it wont be even close to as good as a human at novel tasks.
I think LLMs are a dead end on the path to AGI.
I think hallucinations are a major unsearched gateway to AGI.
Not only humans hallucinate all the time, humans also have persistent hallucinations as evident from the presence of opposing beliefs in various slices of society.
It's actually not very easy to achieve this. I could give a very long winded answer (don't tempt me) but suffice to say it's a resolution problem.
All AI have a fixed resolution on creation. Long running tasks focus on a very particular narrowing space per step, the resolution required for an infinite task is infinite resolution.
No 9s of error will ever fix this.
Funny enough, small animals do this with ease so I strongly disagree the idea that our AI outcompete even small mammals in every way.
More specifically, something like “whats the best brand of phone”. The LLM just summarizes common knowledge. But even a child will grasp some of the differences and have opinions drawn from experience.
Note that this isn’t just an anthro-good argument. AI systems could have experiences and be trained on long duration tasks with memory of what worked and why.
It's obvious to everyone who isn't willfully blind that LLMs aren't truly intelligent, and all the mental gymnastics that people go through to try to portray LLMs as genuinely intelligent is just so tedious.
What's the point of this restriction? It really just presupposes the limitation of LLM, so that any negative points would look moot.
EDIT: Also, I tried to discuss this very specific point w/ GPT, but it didn't really "get" it. 15-year old kids would be able to follow through.
Learn something without a megawatt hour of power.
Read a novel and talk about what it really means.
But don't worry just yet, GPT-4o could not detect the irony on its own either.
Also if you ask them a question they can provide you one answer with very little thinking, and then if that’s not good enough they can devote more time to thinking about the answer before they answer again. They can devote arbitrary levels of thinking to any problem depending on what is needed. They can continuously take in new data and continually update their world view throughout their entire existence based on this new information.
There’s actually a huge list of things current autoregressive approaches to AI cannot do, but they can be hard to describe and people don’t like to talk about them so many people actually don’t understand how limited the current systems are.
Here’s a great video where Yann Lecun talks about the limits of autoregressive approaches to AI with many examples:
The quality of your argument is very low. You didn't even bother to check yourself.
He also says that such a system wouldn’t be familiar with how to actually move through the world because we don’t have good datasets for how to do so. The rest of what I said still stands. These systems aren’t good at things for which we don’t have massive datasets, and they’re not able to devote different amounts of thinking time to different problems.
For reasoning you can write out the logic of your reasons, so there's that. But that's absolutely not required for AGI. People can already go a long way (often further than by reasoning) on intuition alone without being able to explain how they reached their conclusions.