The side effects of spending funds on these mega projects is also something to consider. NASA spending has created a huge pile of technologies that we use day to day: https://en.wikipedia.org/wiki/NASA_spin-off_technologies.
We're a little too early to know if that's the case here too. I do foresee a chance at a reality where AI is a dead end, but after it we have a ton of cheap GPU compute lying about, which we all rush to somehow convert into useful compute (by emulating CPU's or translating traditional algorithms into GPU oriented ones or whatever).
I think we are saying the same thing.i just think the pull back on AI will be dramatic unless something amazing happens very soon.
Why would I pull back?
Perhaps if we used something exotic like solid gold cookware, there might be some amazing benefits that people would love.
But it would be far from practical without being wildly subsidized…
With AI, it feels too much like the “grownups” are acting worse than the kids…
A firm that see's rising operating expenses but no not enough increase in revenue will start to cut back on spending on LLMs and become very frugal (e.g. rationing).
But current gen AIs are like eternal juniors, never quite ready to operate independently, never learning to become the expert that you are, they are practically frozen in time to the capabilities gained during training. Yet these LLMs replaced the first few rungs of the ladder so human juniors have a canyon to jump if they want the same progression you had. I’m seeing inexperienced people just using AI like a magic 8 ball. “The AI said whatever”. [0] LLMs are smart and cheap enough to undercut human juniors, especially in the hands of a senior. But they’re too dumb to ever become a senior. Where’s the big money in that? What company wants to pay for the “eternal juniors” workforce and whatever they save on payroll goes to procuring external seniors which they’re no longer producing internally?
So I’m not too sure a generation of people who have to compete against the LLMs from day 1 will really be producing “so much more” of value later on. Maybe a select few will. Without a big jump in model quality we might see “always junior” LLMs without seniors to enhance. This is not sustainable.
And you enhancing your carpentry skills for your free time isn’t what pays for the datacenters and some CEO’s fat paycheck.
[0] I hire trainees/interns every year, and pore through hundreds of CVs and interviews for this. The quality of a significant portion of them has gone way down in the past years, coinciding with LLMs gaining popularity.
But AI proliferation is not stopping soon, because we've not picked up even the low hanging fruits just yet. Again, even if no new SOTA models were to be trained after today, there's years if not decades of R&D work into how to best use the ones we have - how to harness the big ones, where to embed the small ones, and of course, more fundamental exploration of the latent spaces and how they formed, to inform information sciences, cognitive sciences, and perhaps even philosophy.
And if that runs out or there is an Anti AI Revolution, we can still run those weather models and route planners on the chips once occupied by LLMs - just don't tell the proles that those too are AI, or it's guillotine o'clock again.
I think my sense of "dead end" would entail none of those directions panning out into anything interesting. You would "explore the latent spaces" only to find nothing of value. Embedding the LLM models wouldn't end up doing anything useful for whatever reason, and philosophy would continue on without any change.
Not least because the slower the frontier advances, the cheaper ASICs get on a relative basis, and therefore the cheaper tokens at the frontier get.
We have a massive scaffolding capability overhang, give it ten years to diffuse and most industries will be radically different.
Again, all of this is obvious if you spend 1k hours with the current crop, this isn’t making any capability gain forecasts.
Just for a dumb example, there is a great ChatGPT agent for Instacart, you can share a photo of your handwritten shopping list and it will add everything to your cart. Just following through the obvious product conclusions of this capability for every grocery vendor’s app, integrating with your fridge, learning your personal preferences for brands, recipe recommendation systems, logistics integrations with your forecasted/scheduled demand, etc is I contend going to be equivalent engineering effort and impact to the move from brick and mortar to online stores.
AI (LLM) progress would stop, and then everything people try to do with those last and most capable models would end up uninteresting or at least temporary. That's the world I'm calling a "dead end".
No matter how unlikely you think that is, you have to agree that it's at least possible, right?
I believe that some of my made up examples won’t end up getting built, but my point is that there is _so much_ low hanging fruit like this.
Of course, anything is _possible_, but let’s talk likelihood.
In my forecast the possible worlds where progress stops and then the existing models don’t end up making anything interesting are almost exclusively scenarios like “Taiwan was invaded, TSMC fabs were destroyed, and somehow we deleted existing datacenters’ installed capacity too” or “neo-Luddites take over globally and ban GPUs”, all of this gives sub-1% likelihood.
You can imagine 5-10% likelihood worlds where the growth rate of new chips dramatically decreases for a decade due to a single black-swan event like Taiwan getting glassed, but that’s a temporary setback not a permanent blocker.
Again, I’m just looking at all the things that can obviously be built now, and just haven’t made it to the top of the list yet. I’m extremely confident that this todo list is already long enough that “this all fizzles to nothing” is basically excluded.
I think if model progress stops then everyone investing in ASI takes a big haircut, but the long-term stock market progression will look a lot like the internet after the dot com boom, ie the bloodbath ends up looking like a small blip in the rear view mirror.
I guess, a question for you - how do you think about coding agents? Don’t they already show AI is going to do more than “end up uninteresting”?
The problem with talking likelihood is that it's an interpretation game. I understand you think it's wholly unlikely that it all fizzles out, I could read that from your first post. I hope it's also clear that I do think it's likely.
That's the point where we have to just agree to disagree. We have no rapport. I have no reason to trust your judgment, and neither do you mine.
However I do feel a lot of this comes down to facts about the world now, eg whether Claude Opus is doing anything interesting, which are in principle places where you could provide some evidence or ideas, along the lines of the detail that I gave you.
My read so far is you are just saying “maybe it fizzles out” which is not going to persuade anyone who disagrees. Sure, “maybe”, especially if you don’t put probabilities on anything; that statement is not falsifiable.
> The problem with talking likelihood is that it's an interpretation game
I am open to updating my model in response to a causal argument, if you care to give more detail. I view likelihoods as the only way to make these sorts of conversations concrete enough that anyone could hope to update each other’s model.
I find it interesting that you chose the shopping list and fridge examples, because my view on the whole LLM hype is that 99% of it is a solution looking for a problem, and shopping and the fridge are historically such a commonly advertised area for technologies desparately looking for an actual use case. I don't think fridge content management and shopping plans are actual pain points in most people's lives. It's not something people would see a benefit in if they didn't have to do it manually. And it's an area with a very low tolerance for the systemic unreliability. The guy needed eggs to bake his cake, but the AI got him eggos instead -- et voilà, another person who thinks this whole "smart" technology is shit and won't deal with it anymore.
And so it goes with most AI use cases I've seen so far. In my view the only thing they're good at is fuzzy search. Coding agents are helpful, but in the end, their secret sauce it just that: fuzzy search.
Can fuzzy search be helpful? Yes, even very helpful! "Bigger than the Internet" helpful? I think not.
And even if chatbot LLM's seem to be a dead end, them and other machine learning algo's will be happy to use the data centers to create/discover a lot of stuff.