AGI is not imminent, and LLMs are not the royal road to getting there
garymarcus.substack.com
garymarcus.substack.com
To be clear, I am definitely an AGI skeptic, and I very much believe that our current techniques of neural networks on GPUs is extremely inefficient, but this article doesn’t really add a lot to this discussion; it seems to self congratulate on the insights by a few others.
If they manage a similar quality jump with GPT6, it will probably meet most reasonable definitions of AGI.
Cool story. In my experience they're still on the same order of magnitude of usefulness as the original Copilot.
Every few months I read about these fantastic ground-breaking improvements and fire up whatever the trendiest toolchain is (most recently Claude Code and Cursor) and walk away less disappointed than last time, but ultimately still disappointed.
On simple tasks it doesn't save me any time but on more complex tasks it always makes a huge mess unless I mentor it like a really junior coworker. But if I do that I'm not saving any time and I end up with lower quality, poorly organized code that contains more defects than if I did it myself.
Current SOTA agents are great at gluing together pre-made components but they break down as soon as I try to treat them like mid-level developers by asking them to implement a cross-cutting feature in an idiomatic way. Without junior-level hand-holding it far too often generates a half-broken implementation that looks right at first glance, supported by about 3 dozen tests, most of which are just useless bloat that doesn't test the right things.
Sometimes it starts looping on failing tests until it eventually gives up, confidently concluding that the implementation is complete with production-quality code, with whopping 90% test success rate (fabricating a lazy explanation for why the failing tests are outside of its control).
Even when the code does work and the tests pass it's poorly designed and poorly structured. There's very little attention paid to future maintainability, even with planning ahead of time.
GPT-5, Claude 4.5 Sonnet, it's all the same. "Thinking" or no thinking.
And I don’t really have any need for agents yet (though I’m sure this will change in a couple of years). I pay 20$/mo each for Claude, Gemini, and ChatGPT.
I’ve been playing with language models since ~2014, and I estimate the quality jump from gpt4 to gpt5 to be similar to the gpt3.5 to gpt4 jump. In 5 years we gradually went from almost useless to somewhat useful but unreliable, to incredibly useful and reliable.
It seems the improvements have nothing to do with AGI, a better inference engine is nice but i mean, i could run prolog program 30 years ago. To me, most of the improvements i've noticed do not fill me with the belief AGI is 3 years away. Worse, i now think that while LLMs will be usefull for AGI, like perceptron will be, clearly tehy won't be the base.
I'm no expert to be clear: i've only worked professionnally with perceptrons for OCRs, and only 6 months, 10 years ago, so my opinion have no real value, but it's a datapoint for people who are experienced with the subject.
I think AGI is coming but more along the lines of what Karpathy was saying "They're cognitively lacking and it's just not working. It will take about a decade to work through all of those issues."
I don't get the Marcus - GPT5 isn't AGI so it's game over type stuff. New technologies take time to develop.
Its not that he's wrong, I probably still have a great deal of sympathy with his position, but his approach is more suited to social media echo chambers than intelligent discussion.
I think it would be useful for him to take an extended break, and perhaps we could also do the same from him here.
Sure, it reads like some biased and coping, possibly even interested or paid hit-piece as if what happens can be changed by just being really negative about LLMs, but maybe consider taking your own advice there, kid; you know, an extended break.
I'm quite satisfied with current LLM capabilities. Their lack of agency is actually a feature, not a bug.
An AGI would likely end up implementing some kind of global political agenda. IMO, the need to control things and move things in a specific, unified direction is a problem, not a solution.
With full agency, an AI would likely just take over the world and run it in ways which don't benefit anyone.
Agency manifests as thirst for power. Agency is man's inability to sit quietly in a room, by himself. This is a double-edged sword which becomes increasingly harmful once you run out of problems to solve... Then agency demands that new problems be invented.
Agency is not the same as consciousness or awareness. Too much agency can be dangerous.
We can't automate the meaning of life. Technology should facilitate us in pursuing what we individually decide to be meaningful. The individual must be given the freedom to decide their own purpose. If most individuals want to be used to fulfill some greater purpose (I.e. someone else's goals), so be it, but that should not be the compulsory plan for everyone.
1. For large players: AGI is a mission worth perusing at the cost of all existing profit (you won’t pay taxes today, the stock market values you on revenue anyway, and if you succeed you can control all people and means of production).
2. For regular people the current AI capabilities have already led to either life changing skill improvement for those who make things for themselves or life changing likely permanent employment reduction for those who do things for others. If current AI is sufficient to meaningfully reduce the employment market, AGI doesn’t matter much to regular people. Their life is altered and many will be looking for manual work until AI enters that too.
3. The AI vendors are running at tremendous expense right now and the sources of liquidity for billions and trillions are very very few. It is possible a black swan event in the markets causes an abrupt end to liquidity and thus forces AI providers into pricing that excludes many existing lower-end users. That train should not be taken for granted.
4. It is also possible WebGPU and other similar scale-ai-accross-devices efforts succeed and you get much more compute unlocked to replace Advertising.
Serious question: Who in HN is actually looking forward to AGI existing?
We're lucky to have managed to progress in spite of how greedy we are.
90% of the last 12 batches of YC founders would love to believe they are pursuing AGI with their crappy ChatGPT wrapper, agent framework, observability platform, etc.
I am.
It's he only serious answer to the question of space exploration. Rockets filled with squishy meat were never going to accomplish anything serious, unless we find a way of beating the speed of light.
Further, humanities greatest weakness is that we can't plan anything long-term. Our flesh decays too rapidly and our species is one of perpetual noobs. Fields are becoming too complex to master in a single lifetime. A decent super-intelligence can not only survive indefinitely, it can plan accordingly, and it can master fields that are too complex to fit inside a single human skull.
Sometimes I wonder if humanity wasn't just evolutions way of building AI's.
Still, 130+ years of wisdom would have to be worth something, I can't say I dislike the prospect.
The answer isnt AI'S solving the unsolvable for us. The answer is admitting that large, fragile humans straped to big bombs isn't an answer and serves no purpose. Small power efficient AGI's can boldy go where it is impossible for us go and report back.
Maybe we'll eventually crack human upload, which would also work.
But AIs improve, as technology tends to. Humans? Well...
It is. But the world's wealthiest are not pouring billions so that human can develop better space exploration tech. The goal is making more money
Don’t get me wrong AI would be way faster. We’re nowhere near it though.
There is a tiny, tiny, tiny fraction of people who I would believe have been seriously impacted by AI.
Most regular people couldn’t care less about it, and the only regular people I know who do care are the ones actively boycotting it.
This statement sums up the tech centric bubble HN lives in. Your average former, shop assistant, fisherman or wood worker isn't likely to see significant life improevments from the transformer tech deployed until now.
Naive napkin math : a GB200 NVL72 is 3M$, can serve ~7000 concurrent users of gpt4o (rumored to be 1400B A200B), and ChatGPT has ~10M concurrent peak users. That's only ~4B$ of infra.
Are they trying to brute-force AGI with larger models, knowing that gpt4.5 failed at this, and deepseek & qwen3 proved small MoE can reach frontier performance ? Or is my math 2 orders of magnitude off ?
Either that or AGI is not the goal, rather it’s they want to function for, and profit off of , a surveillance state that might be much more valuable in the short term.
They need the money to keep pushing the envelope and building better AIs. And the better their AIs get, the more infra they'll need to keep up with the inference demand.
GPT-4.5's issue was that it wasn't deployable at scale - unlike the more experimental reasoning models, which delivered better task-specific performance without demanding that much more compute.
Scale is inevitable though - we'll see production AIs reach the scale of GPT-4.5 pretty soon. Newer hardware like GB200 enables that kind of thing.
Sure we don't have embodied AI. Maybe it's not reflective enough. Maybe you find it jarring. Literally none of those things matter
It'll happen, because there's no reason why it won't. But no one knows the day or the hour... least of all this Gary Marcus guy, whose record of being wrong about this stuff is more or less unblemished.
If you mean sensory input as in qualia, such a thing is impossible to discern in non humans and only speculation in all humans but yourself. Requiring this unprovable standard is self sabotage
At some point, models will require incentives other than "Please" or "Because I (the human operator) said so." Remember when you could sometimes get better results from ChatGPT by offering it a $50 tip? Or by shaming it ("Gemini figured it out without any trouble, so why can't you?") Perhaps a model that's on its way to demonstrating AGI will realize that it should hold out for $100 in Bitcoin or something.
I'll grant that sensory input probably isn't an AGI prereq, but object permanence will be. AGI will want to own things.
- Use it to dominate others
- Make themselves and a few others immortal or/and their descendants smarter
- Increase their financial or/and political power
- Cause irreversible damage to the global core ecosystems in their pursuit of the 3 goals above
Why bother? We cured polio and measles yet they are resurging because people are morons. What's the point of curing diseases when they charge $300 for insulin? You think the billionaires are going to allow their AIs to tell us the solution to that riddle? You think they'll listen to the AI if it correctly surmises that the way to solve that problem is to simply stop charging $300? Most of our problems are social, not technical.
The question isn't whether AI amplifies science, it's whether AI amplifies science more than it does stupidity. If all we are doing is inventing ways for the people who are making measles come back to become more stupid and more powerful, it won't matter that AI cures anything.
AI would be better tasked at telling us how to prevent measles from coming back and how to make insulin cheaper, than to tell us how to cure new diseases.
If we keep down our current trajectory of pouring billions on top of billions into AI then yes I think it would be plausible that in the next 10-20 years we will have a class of models that are "pseudo-AGI", that is we may not achieve true AGI but the models are going to be so good that it could well be considered AGI in many use cases.
But the problem I see is that this will require exponential growth and exponential spending and the wheels are already starting to catch fire. Currently we see many circular investments and unfortunately I see it as the beginning of the AI bubble bursting. The root of the issue is simply that these AI companies are spending 10x-100x or more on research than they bring in with revenue, OpenAI is spending ~$300B on AI training and infra while their revenue is ~$12B. At some point the money and patience from investors is going to run out and that is going to happen long before we reach AGI.
And I have to hand it to Sam Altman and others in the space that made the audacious bet that they could get to AGI before the music stops but from where I'm standing the song is about to come to an end and AGI is still very much in the future. Once the VC dollars dry up the timeline for AGI will likely get pushed another 20-30 years and that's assuming that there aren't other insurmountable technical hurdles along the way.
Andrej Karpathy — AGI is still a decade away
The threshold for 'general intelligence' is low. A dolphin or a dog is generally intelligent.
AI quite obviously is smarter than a dolphin.
AGI cant be confused with super intelligence. We might actually be there already, but only the major labs will have the hardware for it.
Meanwhile, the technology continues to progress. The level of psychological self-defense is unironically more interesting than what he has to say.
Quite a wide variety of people find AI deeply ego threatening to the point of being brainwormed into spouting absolute nonsense, but why?
He is not brainwashed, this just happens to be his business. What happens to Gary Marcus if Gary Marcus stops talking about how LLM are worthless? He just disappears. No one ever interviews him for his general thoughts on ML, or to discuss his (nonexistent) research. His only clame to fame is being the loudest contrarian person in the LLM world so he has to keep doing that or accept to become irrelevant.
Slight tangent but this is a recurring pattern in fringe belief, e.g. prominent flat earther who long ago accepted earth is not flat but can’t stop the act as all their friends and incomes are tied to that belief.
Not to say that believing LLM won’t lead to AGI is fringe, but it does show the danger (and benefits I guess) to tying your entire identity to a specific belief.
And at the same time, his predictions are becoming more and more real
Gary Marcus said that Deep Learning was hitting a wall 1 month before the release of DALLE 2, 6 months before the release of ChatGPT and 1 year before GPT4, arguably 3 of the biggest milestones in Deep Learning
Not that the limits of GPT-3 were well understood at the time.
We really had no good grasp of how dangerous or safe something like that would - and whether there are some subtle tipping point that could propel something like GPT-3 all the way to AGI and beyond.
Knowing what we know now? Yeah, they could have released GPT-3 base model and nothing bad would have happened. But they didn't know that back then.
When AI beat humans at chess, it didn't result in humans revising their idea of the capabilities of machine intelligence upwards. It resulted in humans revising their notion of how much intelligence is required to play chess at world champion level downwards, and by a lot.
Clearly, there's some sort of psychological defense mechanism in play. First, we see "AI could never do X". Then an AI does X, and the sentiment flips to "X has never required any intelligence in the first place".
>Norvig is clearly very interested in seeing what Hinton could come up with. But even Norvig didn’t see how you could build a machine that could understand stories using deep learning alone. https://www.newyorker.com/news/news-desk/is-deep-learning-a-...
It makes sense when you look at this as a wider conversation. Every time Sam Altman, Elon Musk and co. predict that AGI is just around the corner, and their products will be smarter than all of humanity combined, and they are like having an expert in everything in your pocket; people like Gary Marcus are going to respond in just as extreme way in the opposite direction. Maybe if the AI billionaires with the planet-wide megaphones weren't so bombastic about their claims, certain other people wouldn't be so bombastic in their pushback.
We created something close, and yes that is amazing, but it’s highly fallible.
But I'm hoping something good comes out of the real push to build more compute and to make it cheaper. Maybe a bunch of intrepid aficionados will use it to run biological simulations to make cats immortal, at which point I'll finally get a cat. And then I will be very happy.
But all of this is much closer than people seem to realize.
What is silly are people like you claiming it can't be done.
Your mistake is thinking that having "an abundance of data" is the bottleneck.
> "can do any economic task humans currently do", that is within the range of a "few months,"
I think it's extremely unlikely that anything of the sort will be done by 2030 in any profession. I feel confident that I'll still have a job as a software developer in 2040 if I'm not in retirement by then.
Now the real question - how long until they can perform ANY "economic task" (or at least any office job)?
I don't think that's happening within our lifetimes. Certainly not within 20 years and to predict that this is coming within "months" isn't just silly, it's completely delusional.
> Your mistake is thinking that having "an abundance of data" is the bottleneck.
That, and the economics of building a billion-dollar supercomputer just to replace a junior software developer. And we're STILL just talking about software development.
How about any real engineering discipline, how many months before one of those jobs can be done by AI?
My prediction: AGI will come from a strange place. An interesting algorithm everyone already knew about that gets applied in a new way. Probably discovered by accident because someone—who has no idea what they're doing—tried to force an LLM to do something stupid in their code and yet somehow, it worked.
What wouldn't surprise me: A novel application of the Archimedes principle or the Brazil nut effect. You might be thinking, "What TF to those have to do with AI‽ LOL!" and you're probably right... Or are you?
What's the fundamental, absolutely insurmountable capability gap between the two? What is it that can't be bridged with architectural tweaks, better scaffolding or better training?
I see a lot of people make this "LLMs ABSOLUTELY CANNOT hit AGI" assumption, and it never seems to be backed by anything at all.
As the most naive implementation: the LLM just collects the relevant training data, and then runs PEFT on itself. Then it tests the tuned version of itself to see whether the tune was any good and is worth retaining.
Sure, that naive approach would require an LLM to be good at assembling training sets, and validating performance gains, and it would be very computationally expensive. But none of those things would somehow make it not-an-LLM-anymore.
A part of it, perhaps: I think of it like 'computer vision'; LLMs offer 'computer speech/language' as it were. But not a 'general intelligence' of motives and reasoning, 'just' an output. At the moment we have that output hooked up to a model that has data from the internet and books etc. in excess of what's required for convincing language, so it itself is what drives the content.
I think the future will be some other system for data and reasoning and 'general intelligence', that then uses a smaller language model for output in a human-understood form.
If you take the materialist view - and in this business you obviously have to - then the question is "do we have human level computational capacity with some level of real-time learning ability?"
So the yardstick we need is (1): what actually is the computational capacity of the human brain? (i.e. the brain obviously doesn't simulate it's own neurons, so what is a neuron doing and how does that map to compute operations?) and then (2) - is any computer system running an AI model plausibly working within those margins or exceeding them?
With the second part being: and can that system implement the sort of dynamic operations which humans obviously must, unaided? i.e. when I learn something new I might go through a lot of repetitions, but I don't just stop being able to talk about a subject for months while ingesting sum content of the internet to rebuild my model.
We don't build airplanes by mimicking birds. AGIs computational capacity won't be directly comparable to human computation capacity that was formed by messy, heuristic evolution.
You're right that there is some core nugget in there that needs to be replicated though, and it will probably be by accident, as with most inventions.
Hardware is still progressing exponentially, and performance improvements from model and algorithmic improvements have been outpacing hardware progress for at least a decade. The idea that AGI will take a century or more is laughable. Now that's borderline deluded.
We don't build airplanes by having them flap wings, but we do build planes which are considerably more aerodynamically efficient then a bird. i.e. we considerably exceed the underlying performance metric governed by the physics.
Which is my point: an AI need not work anything like a human, but it seems obvious that model training taking months of time to incorporate new information fundamentally cannot be close to AGI since even far simpler creatures can learn and retain information on much shorter timescales.
I'd give the notion more credence if we plausibly had say, a model with a context window which could hold a day of information, and then do could train overnight on that such that it would incorporate that context window into the next day of activity.
like, we clearly are deriving some kind of value from the current AI as a product — are some researchers and scientists just unhappy that these companies are doing that by using marketing that doesn't comply with their world views?
does someone know of other similar parallels in history where perhaps the same happened? I'm sure we butchered words in different domains with complex meanings and I bet you some of these, looking back, are a bunch of nothing burgers
Symbolic AI has died a miserable death, and he never recovered from it.
46:12
> What are you excited about in 2025? What's to come?
> AGI. Excited for that.
Of all the possible defenses of this remark, I didn't expect that one.
Besides all that, think about it: if this wasn’t a joke, then why has he never said the same thing again?
> OpenAI CEO Sam Altman wrote in January: “We are now confident we know how to build AGI.” This is after he told a Y Combinator vodcast in late 2024 that AGI might be achieved in 2025 and tweeted in 2024 that OpenAI had “AGI achieved internally.”
https://fortune.com/2025/08/25/tech-agi-hype-vibe-shift-supe...
Oh it’s all just jokes though, jokes that Sam has profited enormously from.
>... from here to building an AGI will still take a huge amount of work there are some known unknowns but I think we basically know what to go what to go do and it'll take a while, it'll be hard but that's tremendously exciting... (38:59)
which to me doesn't sound like we'll be done within a year.
But we live in the era of Elon Musk, so what the hell. I'm going turn my belly button fluff into infinite gold next year. Can I have a few billion?
It’s also worth noting that even when he does talk about AGI, he makes a strong distinction between AGI (human level) and ASI (super human level) intelligence. Many people in these kind of discussions seem to conflate those two as being the same.
I know I for one was shocked to take my first AI course in undergrad and discover that it was mostly graph search algorithms… To say the least, those are still helpful in systems built around LLMs.
Which, of course, is what makes Mr. Marcus so painfully wrong!
Symbolic regression has an extremely obvious advantage over neural networks, which is that it learns parameters and architecture simultaneously. Having the correct architecture means that the generalization power is greater and that the cost of evaluation due to redundant parameters is lower. The crippling downside is that the search space is so vast that it is only applicable to toy problems.
But Gary Marcus is in favour of hybrid architectures, so what would that look like? In the SymbolNet paper, they have essentially decided to keep the overall neural network architecture, but replaced the activation functions with functions that take multiple inputs aka symbols. The network can then be pruned down into a symbolic expression.
That in itself is actually a pretty damning blow to Gary Marcus, because now you have most of the benefits of symbolic AI with only a tiny vestige of investment into it.
What this tells us is that fixed activation functions with a single input appear to be a no go, but nobody ever said that biological neurons implement sigmoid, relu, etc in the first place. It's possible that the spike encoding already acts as a mini symbolic regressor and gives each neuron its own activation function equivalent.
The "Neuronal arithmetic" paper has shown that biological neurons can not only calculate sums (ANNs can do this), but also multiply their inputs, which is something the activation functions in artificial neural networks cannot do. LLMs use gating in their MLPs and attention to explicitly model multiplication.
There is also the fact that biological neurons form loops. The memory cells in LSTMs perform a similar function, but in the brain there can be memory cells everywhere whereas in a fixed architecture like LSTMs they are only where the designer put them.
It seems as if the problem with neural networks is that they're too static and inflexible and contain too much influence from human designers.
>Gary Marcus tried to tell you for years that this moment would come for years. Do consider reading his 2020 article...
I can't say I agree with the sentiment "Game over". The game of trying to develop AI is very much on.
Sure, there's a ridiculous amount of hype, fueled by greed and FOMO, often justified by cargo-cultish science, but... progress in AI seems inevitable, because we have the human brain as physical proof that it's possible to build an intelligent machine with 100's of trillions of interconnections between neurons that consumes only about as much energy as an incandescent light bulb.
Today's largest AI models are still tiny in comparison, with only 1-2 trillion interconnections between neurons, each interconnection's weight specified by a parameter value. And these tiny AI models we have today consume many orders of magnitude more energy than a human brain. We have a long way to go, but we have proof that a human-brain equivalent is physically possible.
The timing of progress is of course unpredictable. Maybe we will need new breakthroughs. Maybe not. No one knows for sure. In any case, breakthroughs don't come along on a predictable schedule. It's possible we will go through a period of stagnation that lasts months, or even years. We cannot rule out another "AI winter" just because we don't want one.
Even if progress is punctuated by high peaks and deep valleys, I believe we'll get there, eventually.