Comment by ComplexSystems | Hacker News Reader

Comment by ComplexSystems | Hacker News Reader

I think the article makes decent points but I don't agree with the general conclusion here, which is that all of this investment is wasted unless it "reaches AGI." Maybe it isn't necessary for every single dollar we spend on AI/LLM products and services to go exclusively toward the goal of "reaching AGI?" Perhaps it's alright if these dollars instead go to building out useful services and applications based on the LLM technologies we already have.

The author, for whatever reason, views it as a foregone conclusion that every dollar spent in this way is a waste of time and resources, but I wouldn't view any of that as wasted investment at all. It isn't any different from any other trend - by this logic, we may as well view the cloud/SaaS craze of the last decade as a waste of time. After all, the last decade was also fueled by lots of unprofitable companies, speculative investment and so on, and failed to reach any pie-in-the-sky Renaissance-level civilization-altering outcome. Was it all a waste of time?

It's ultimately just another thing industry is doing as demand keeps evolving. There is demand for building the current AI stack out, and demand for improving it. None of it seems wasted.

That’s not what he’s saying, the investors are the ones who have put trillions of dollars into this technology on the premise that it will achieve AGI. People like Sam Altman and Marc Andreesen have been going into podcasts saying AGI is imminent and they’re going to automate every job.

The author did not say every dollar was wasted, he said that LLMs will never meet the current investment returns.

It’s very frustrating to see comments like this attacking strawmans and setting up Motte and Bailey arguments every time there’s AI criticism. “Oh but LLMs are still useful” and “Even if LLMs can’t achieve AGI we’ll figure out something that will eventually.” Yes but that isn’t what Sam and Andreesen and all these VCs have been saying, and now the entire US economy is a big gamble on a technology that doesn’t deliver what they said it would and because the admin is so cozy with VCs we’re probably all going to suffer for the mistakes of a handful of investors who got blinded by dollar signs in their eyes.

The author quite literally says that the last few years were a "detour" that has wasted a trillion dollars. He explicitly lists building new LLMs, building larger LLMs and scaling LLMs as the problem and source of the waste. So I don't think I am strawmanning his position at all.

It is one thing to say that OpenAI has overpromised on revenues in the short term and another to say that the entire experiment was a waste of time because it hasn't led to AGI, which is quite literally the stance that Marcus has taken in this article.

> The author, for whatever reason, views it as a foregone conclusion that every dollar spent in this way is a waste of time and resources

This is a strawman, the author at no point says that “every dollar is a waste.”

He quite literally says that the dollars spent on scaling LLMs in the past few years are a waste.

Yes and you’re portraying it as “every dollar was wasted” and then arguing against that exaggerated claim which was never made.

You are making the same strawman attack you are criticising.

The dollars invested are not justified considering TODAYs revenues.

Just like 2 years ago people said NVIDIA stock prices was not justified and a massive bubble considering the revenue from those days. But NVIDIA revenues 10xed, and now the stock price from 2 years ago looks seriously underpriced and a bargain.

You are assuming LLM revenues will remain flat or increase moderately and not explode.

You seem like someone who might be interested in my nuclear fusion startup. Right now all we have is a bucket of water but in five years that bucket is going to power the state of California.

It's the nature of bubbles to overpromise and underdeliver.

So - do Altman and Andreesen really believe that, or is it just a marketing and investment pitch?

A reminder that OpenAI has its own explicit definition of AGI: "Highly autonomous systems that outperform humans at most economically valuable work."

The MS/OAI agreement quantifies that as "profits of $100bn/yr."

This seems rather stupid to me. If you can get multiple AGIs all generating profits of $100bn a year - roughly half an Apple, or two thirds of a Meta - you no longer have anything recognisable as a pre-AI economy, because most of the white collar population is out of work and no longer earning anything. And most of the blue collar population that depends on white collar earnings is in the same boat.

So you have to ask "Profits from doing what, and selling to whom?"

The Altman pitch seems to be "Promise it first, collect investor cash, and worry about the rest later."

> do Altman and Andreesen really believe that, or is it just a marketing and investment pitch?

As for Andreessen, I don't think he even cares. As the author writes:

"for the venture capitalists that have driven so much of field, scaling, even if it fails, has been a great run: it’s been a way to take their 2% management fee investing someone else’s money on plausible-ish sounding bets that were truly massive, which makes them rich no matter how things turn out"

VCs win every time. Even if it's a bubble and it bursts, they still win. In fact, they are the only party that wins.

Heck, the bigger the bubble, the more money is poured into it, and the bigger the commissions. So VCs have an interest in pumping it up.

Those billions of dollars spent on data centres could instead have hypothetically been spent on researchers with relatively modest computing resources. A few hundred billion dollars will buy you a lot of researchers.

It's not about "every dollar spent" being a waste of time, it's about acknowledging the reality of opportunity cost. Of course, no one in any movement is likely to listen to their detractors, but in this case the pioneers seem to agree.

https://www.youtube.com/watch?v=DtePicx_kFY https://www.bbc.com/news/articles/cy7e7mj0jmro

I think there is broad agreement that new models and architectures are needed, but I don't see it as a waste to also scale the stack that we currently have. That's what Silicon Valley has been doing for the past 50 years - scaling things out while inventing the next set of things - and I don't see this as any different. Maybe current architectures will go the way of the floppy disk, but it wasn't a waste to scale up production of floppy disk drives while they were relevant. And ChatGPT was still released only 3 years ago!

And notably, Marcus has been banging this drum for years. Even this article points back to articles he wrote years ago suggesting deep learning was hitting the wall... With GPT 3....

It's sour grapes because the methods he prefers have not gotten the same attention (hah...) or funding.

He's continuing to push the ludicrous Apple "reasoning paper" that he described as a "knockout blow for LLMs" even though it was nothing of the sort.

With each of his articles, I usually lose more respect for him.

Reply on news.ycombinator.com