(See observation 1 for context) https://blog.samaltman.com/three-observations
I feel this is way out of line given the resources and marginal gains. If the claimed scaling laws don’t hold (more resources = closer to intelligence) then LLMs in their current form are not going to lead to AGI.
It always seemed to me a wild leap to assume that LLMs in their current form would lead to AGI. I never understood the argument.
In nutshell: https://xkcd.com/605/
Grug version: Man sees exponential curve. Man assumes it continues forever. Man says "we went from not flying to flying in few days, in few years we will spread in observable universe. "
That may happen in 200 years. Geologically as you zoom out, the difference is negligible.
This is way way less than the observable universe.
(I've seen this 'enough money ought to surmount any barrier' take a few times, usually to reject the idea that we might not find any path to AGI in the near future.)
The delicious irony is that we know how to solve climate change.
We’re simply not doing enough about it.
Only a few edge cases remain.
Same for dark matter. There are a few hypothesis, but all in all they are pretty simplistic cases.
We know the particles, the forces, there is not really “new physics” to be discovered here.
All the interactions anybody can encounter in their lives is fully understood.
I suspect it is an intentional result of deceptive marketing. I can easily imagine an alternative universe where different terminology was used instead of "AI" without sci-fi comparisons and barely anyone would care about the tech or bother to fund it.
I mean, certainly people like Sam Altman was pushing it hard, so it’s easy to understand how an outside observer would be confused.
But it also feels like a lot of VCs and AI companies have staked several hundreds of billions of dollars on that bet, and I’m still… I just don’t see why the inside players—that should (and probably do!) have more knowledge than me—see. Why are they dumping so much money into this bet?
The market for LLMs doesn’t seem to support the investment, so it feels like they must be trying to win a “first to AGI” race.
Dunno, maybe the upside of the pretty unlikely scenario is enough to justify the risk?
“A person is smart. People are dumb, panicky dangerous animals and you know it.”
If AGI wants to hit human level intelligence, I think it’s got a long way to go. But if it’s aiming for our collective intelligence, maybe it’s pretty close after all…
It is still interesting tech. I wish it were being used more for search and compression.
Sam Altman is a very good hype man. I don’t think anyone on the inside genuinely thinks LLMs will lead to AGI. Ed Zitron has been looking at the costs vs the revenue in his newsletter and podcast and he’s got me convinced that the whole field is a house of cards financially. I already considered it much overblown, but it’s actually one of the biggest financial cons of our time, like NFTs but with actual utility.
If you find yourself agreeing, I highly recommend subscribing to his newsletter.
They only need to last until the exit (potentially next round).
> The market for LLMs doesn’t seem to support the investment
i.e. it doesn't matter as long as they find someone else to dump it to (for profit).
So yep, a lot of time, they bet on trends. Cryptocurrencies, NFTs, several waves of AI. The question is just the acquisition or IPO price.
I don't doubt that some VCs genuinely bought into the AGI argument, but let's be frank, it wasn't hard to make that leap in 2023. It was (and is) some mind-blowing, magical tech, seemingly capable of far more than common sense would dictate. When intuition fails, we revert to beliefs, and the AGI church was handing out brochures...
It...does seem hard to make that leap to me. I mean, again, to a casual and uncritical outside observer who is just listening to and (in my mind naively) trusting someone like Sam Altman, then it's easy, sure.
But I think for those thinking critically about it... it was just as unjustified a leap in 2023 as it is today. I guess maybe you're right, and I'm just really overestimating the number of people that were thinking critically vs uncritically about it.
I mean, see also, AR/VR/Metaverse. My suspicion is that, for the like of Google and Facebook, they have _so much money_ that the risk of being wrong on LLMs exceeds the risk of wasting a few hundred billion on LLMs. Even if Google et al don’t really think there’s anything much to LLMs, it’s arguably rational for them to pump the money in, in case they’re wrong.
That said, obviously this only works if you’re Google or similar, and you can take this line of reasoning too far (see Softbank).
People were declaring ELIZA was intelligent after interacting with it and ELIZA is barely a page of code.
I think what they should be saying is - from a software stack standpoint, current tech unlocks AGI if it can be sped up significantly. So what we’re really waiting for are more software and hardware breakthroughs that make their performance many orders of magnitude quicker.
Consider also that any advanced language model already surpasses individual human knowledge, since each model compresses and synthesizes the collective insights, wisdom, and information of human civilization.
Now imagine a frontier model with agency, capable of deliberate, reflective thought: if it could spend an hour thinking, but that hour feels instantaneous to us, it would essentially match or exceed the productivity and capability of a human expert using a computer. At that point, the line between current AI capabilities and what we term AGI becomes indistinguishable.
In other words: deeper reflection combined with computational speed means we’re already experiencing AGI-level performance—even if we haven’t fully acknowledged or appreciated it yet.
For example John says "Please ask Jane to buy me an ice-cream" and the AI might be able to do that. If she doesn't, John can ask it to coerse her.
Granted that latter delta is much harder to measure, but history has shown repeatedly that that delta is always orders of magnitude bigger than we think it is when $GOAL=AGI.
That’s what we put in our 2017 paper.
It does mean that there is a simple way to keep building smarter LLMs.
I have never seen a clear definition of what AGI is and what it means to achieve it
In truth, basically everything in reality settles towards an equilibrium. There is no inevitable ultraviolet catastrophe, free energy machine, Malthusian collapse. Moore's law had a good run, but frequencies stopped improving twenty years ago and performance gains are increasingly specific and expensive. My car also accelerates consistently through many orders of magnitude, until it doesn't. If throwing more mass at the problem could create general superintelligence and do so economically to such strong advantages, then why haven't biological neural networks, which are vastly more capable and efficient neuron-for-neuron than our LLMs, already evolved to do so?
"Man selling LLMs says LLMs will soon take over the world, and YOU too can be raptured into the post-AI paradise if you buy into them! Non-believers will be obsolete!" No, he hasn't studied enough neuroscience nor philosophy to even be able to comment on how human intelligence works, but he's convinced a lot of rich people to give him more money than you can even imagine, and money is power is food for apes is fancy houses on TV, so that must mean he's qualified/shall deliver us, see...
> The intelligence of an AI model roughly equals the log of the resources used to train and run it.
Pretty sure how log curves work is you add exponentially more inputs to get linear increases in outputs. That would mean it's going to get wildly more difficult and expensive to get each additional marginal gain, and that's what we're seeing.
Essentially the claim is that it gets exponentially cheaper at the same rate the logarithmic resource requirements go up. Which leads to point 3 - linear growth in intelligence is expected over the next few years and AGI is a certainty.
I feel that's not happening at all. This costs 15x more to run compared to 4o and is marginally better. Perhaps the failing of this prediction is on the cost side but regardless it's a failing of the prediction for linear growth in intelligence. Would another 15x resources lead to another marginal gain? Repeat? At that rate of gain we could turn all the resources in the world to this and not hit anywhere near AGI.
Third point, I think, is that the intelligent work that the models can do will increase exponentially because they get cheaper and cheaper to operate as the get more and more capable.
So I think GPT4.5 is in the short term more expensive (they have to pay their training bill) but eventually this being the new floor is just ratcheting toward the nerd singularity or whatever.
When scaling saturates, less computationally expensive models would benefit more.
4o has been very heavily optimized though. I'd say a more apples-to-apples comparison would be to original gpt-4, which was $30/$60 to 4.5's $75/$150. Still a significant difference, but not quite so stark.
We now have llama 3.3 70B, which by most metrics outperforms the 405B model without further scaling, so it’s been my assumption that scaling is dead. Other innovations in training are taking the lead. Higher volumes of low quality data aren’t moving the needle.
Linear increases in intelligence yield exponential increases in economic gains.
Therefore, exponential increases in inputs yield exponential increases in economic gains.
Left as an exercise for the reader is to determine who will capture most of those economic gains.
Seems on par with what expected, but there's lots of unknowns
I think scaling LLMs with their current architecture has an inherent S-curve. Now comes the hard part to develop and manage engineering in the space with ever increasing complexity. I believe there is an analogy to efficiency of fully connected networks versus structured networks. The latter tend to perform more efficiency, to my understanding, and my thanks for inspiring yet another question for my research list.
This s-curve good, though. Helps us to catch up and use current tech without it necessarily being obselete the second after we build it or read about it. And as the current generation of AI can improve productivity in some sectors, perhaps 40% to 60% in my own tasks and what I have read from Matt Baird (LinkedIn economist) and Scott Cunningham. This helps us push back against Baumol's cost disease.
The scaling law only states that more resources yield lower training loss (https://en.wikipedia.org/wiki/Neural_scaling_law). So for an LLM I guess training loss means its ability to predict the next token.
So maybe the real question is: is next token prediction all you need for intelligence?
And before we go to “the token predictor could compensate for that…” maybe we should consider that the reason this is the case is because intelligence isn’t actually something that can be modeled with strings/tokens.
So basically, I don't even believe in AGI. Either we have it, relative to how we would have described it, or it's a goal post that keeps moving that we'll never reach.
That's a pretty fundamental level of base reasoning that any truly general intelligence would require. To be general it needs to apply to our world, not to our pseudo-linguistic reinterpretation of the world.
“9.9 is larger than 9.11.
This is because 9.9 is equivalent to 9.90, and comparing 9.90 to 9.11, it’s clear that 90 is greater than 11 in the decimal place.”
Exodus 9.9 is less than Exodus 9.11.
Linux 9.9 is less than Linux 9.11
But if we're already at your AGI goalpost, I think you could stand to move it quite a ways the other direction.
Perhaps it will evolve into something useful but at present it is nowhere near independent intelligence which can reason about novel problems (as opposed to regurgitate expected answers). On top of that Sam Altman in particular is a notoriously untrustworthy and unreliable carnival barker.
Yes, if you just showed them a demo its super impressive and looks like an AGI. If you let a lawyer, doctor or even a programmer actually work deeply with it for a couple of months I don't think they would call it AGI, whatever your definition of AGI is. It's a super helpful tool with remarkable capabilities but non factuality, no memory, little reasoning and occasional hallucinations make it unreliable and therefore non AGI imo.
It still goes round in circles and makes things up (which it later "knows" the right answer to).
None of that is anywhere near AGI as it's not general intelligence.
LLMs are really only passable when either the topic is trivial, with thousands of easily Googleable public answers, or when you yourself aren't familiar with the topic, meaning it just needs to be plausible enough to stand up to a cursory inspection. For anything that requires actually integrating/understanding information on a topic where you can call bull, they fall apart. That is also how human bullshit artists work. The "con" in "conman" stands for "confidence", which can mask but not stand in for a lack of substance.