He is pointing out that the current costs to create the data centres means you will never be able to make a profit to cover those costs. $800 Billion just to cover the interest.
OpenAI is already haemorrhaging money and the space data centres has already been debunked. There is even a recent paper that points out that LLMs will never become AGI.
The article also finishes out with some other experts giving the same results.
[edit] Fixed $80 to $800
can you share a link?
The core piece as quoted from the abstract: "AGI predictions fail not from insufficient compute, but from fundamental misunderstanding of what intelligence demands structurally."
Then goes in detail as to what that is and why LLMs don't fit that. There are plenty other similar papers out there.
I have read it. It is nothing new on the subject, but it was just the recent paper I saw on HN and the person was asking for the link.
The crux is an LLM is and can never be intelligent in the sense of an AGI. It is easier to think of it as a way to store and retrieve knowledge.
Even if I did read it, I have no hope of understanding if it has made a fundamental mistake because I don't have the subject matter expertise either.
(I imagine it has made a fundamental mistake anyway: for LLMs to be useful progress toward AGI they don't have to be a feasible way to create AGI by themselves. Innovation very often involves stepping through technologies that end up only being a component of the final solution, or inspiration for the final solution. This was always going to be an issue with trying to prove a negative.)
It was a paper posted on HN a few days ago and someone asked for the evidence of my statement. I supplied it.
Now if they actually read it and disagreed with what it was saying, I'd be more than happy to continue the conversation.
Dismissing it just because you don't understand is a terrible thing to do to yourself. It's basically sabotaging your intelligence.
Sometimes papers are garbage, but you can only make that statement after you have read/understood it.
Use an LLM if you want.
People don't assume LLM will be AGI, people assume that World Models will lead us to AGI.
I personally never asumed LLM will become AGI, i always assumed that LLM broke the dam for investment and research into massivce scale compute ML learning and LLMs are very very good in showing were the future goes because they are already so crazy good that people can now imagine a future were AGI exists.
And that was very clear already when / as soon as GPT-3 came out.
The next big thing will probably be either a LOT more RL or self propelling ai architecture discovery. Both need massive compute to work well but then will potentially provide even faster progress as soon as humans are out of the loop.
I wish that was true.
> people assume that World Models will lead us to AGI.
Who are these people? There is no consensus around this that I have seen. You have anything to review regarding this?
> as soon as GPT-3 came out.
I don't think that was true at all. It was impressive when it came out, but people in the field clearly saw the limitations and what it is.
RL isn't magical either. Google AlphaGo as an example often required human intervention to get the RL to work correctly.
Regarding world models: All the big ones. LeCun, Demis Hassabis, Fei-Fei Li too. And they are all working on it.
LLMs will definitly play some type of role in AGI. After all you can ask an LLM already a lot of basic things like 'what are common tasks to make a tea'. A type of guide, long term fact memory or whatever this can be called.
You should research it and not just read news articles. RL did not work and required human intervention numerous times before it got close to what it is now.
There are indications that Open AI is doing this but nothing official as far as i know and i have not heard anything from Anthropic.