All of science is getting harder as the easiest discoveries are all pretty much behind us.
LLMs were a breakthrough I didn't expect and it's likely the last one we'll see in our lifetime.
LLMs were a breakthrough I didn't expect and it's likely the last one we'll see in our lifetime.
Either way this is also opinion based.
There hasn't been a revolutionary change in technology in the last 20 years. I don't consider smart phones to be revolutionary. I consider going to the moon revolutionary and catching a rocket sort of revolutionary.
Actually I take that back I predict mars as a possible break through along with LLMs, but we got lucky with musk.
catching a rocket is very impressive, but its just a lower cost method for earth orbit. it does unlock megaconstellations tho
AI is the step function change. The irony is that it became so pervasive and intertwined with slop people like you forget that what it does now (write all code) was unheard of just a couple years ago. ai surpassed the hype, now it’s popular to talk shit about it.
For decades, progress mostly shifted physical constraints or communication bandwidth. Faster chips, better networks, cheaper storage. Those move slopes, not discontinuities. Humans still had to think, reason, design, write, debug. The bottleneck stayed human cognition.
LLMs changed that. Not marginally. Qualitatively.
The input to the function used to be “a human with training.” The output was plans, code, explanations, synthesis. Now the same class of output can be produced on demand, at scale, by a machine, with latency measured in seconds and cost approaching zero. That is a step change in effective cognitive throughput.
This is why “video calling another continent” feels incremental. It reduces friction in moving information between humans. AI reduces or removes the human from parts of the loop entirely.
You can argue about ceilings, reliability, or long term limits. Fine. But the step already happened. Tasks that were categorically human two years ago are now automatable enough to be economically and practically useful.
That is the function. And it jumped.
However, from your later comments, it sounds as though you feel the only operating definition of a "breakthrough" is a change inducing a rapid rise in labor extraction / conventional productivity. I could not disagree more strongly with this opinion, as I find this definition utterly defies intuition. It rejects many, if not most, changes in scientific understanding that do not directly induce a discontinuty in labor extraction. But admittedly if one restricts the definition of a breakthrough in this way, then, well, you're probably about right. (Though I don't see what Mars has to do with labor extraction.)
To which AI is the only technology that has enough distance to be classified as a “breakthrough”.
A description that matches reality is realist, not pessimist.
This means that most people who you would term as "realists" are likely optimists and not realists at all.
It will give it to you.
If you mean nearest neighbours search like autocorrect then LLMs are extrapolative.
You can easily generate combinations not seen before. I mean you can prove this with parametric prompting.
Like "Generate a poem about {noun} in {place} in {language}" or whatever. This is a simplistic example but it doesn't take much to come up with a space that has quadrillion of possibilities. Then if you randomly sample 10 and they all seem to be "right" then you have proven it's not pure neighbour recall.
Same is true of the image generators. You can prove its not memorizing because you can generate random varients and show that the number of images realizable is more than the training data possibly contains.
If you mean on the underlying manifold of language and ideas. Its definitely interpolation, which is fundamentally a limitation of what can be done using data alone. But I know this can be expanded over iteration (I have done experiments related to this). The trick to expanding it actually running experiments/simulation on values at the boundry of the manifold. You have to run experiments on the unknown.
But I get it, the interpolation you’re talking about is limited. But I think you missed this insight: human interpolation is limited too. In the short term everything we do is simply recombination of ideas as you put it.
But that’s the short term. In the long term we do things that are much greater. But I think this is just an aggregation of small changes. Change the words in a poem 5000 times: have the LLM do the same task 5000 times. Let it pick a random word. The result is wholly original. And I think in the end this what human cognition is as well.
Even if an LLM came up with a theory of quantum gravity in some random chain of thought via chance, once the context is wiped everything is gone.
Expanding the frontier of knowledge (true extrapolation) requires iteration and layering of sinpler ideas. If you loose the layers and have to start from scratch every time then you fundamently will never move further out then what you already know (the interpolation).
You missed my point. I'm saying humans have finite context windows as well.
Look at how claude keeps passing it's context window down the chain. It creates a summary. It can spend thousands of tokens to coalesce on a conclusion, and only that conclusion needs to be passed on to the next context window. The research can be tossed. That's how human discovery works. We don't need the whole context window, we produce major discoveries because we pass the conclusion down the chain.
LLMs can do it too. We just never fully tried it.