If anything performance should get better with time
This is often the case with these types of technologies.
To not look far - gpt3.5 turbo.
You Z80 computer cost $700 in the lat 70's...they're now in sub-$1 embedded controllers.
10% here and there is very small compared to the literal orders magnitude improvements during the reign of Moore's Law.
I don't really see anything like that here.
I can't confirm it, but I noticed this comment says "gpu tech has beat Moore’s law for DNNs the last several years":
For a long time, GPU hardware basically became more powerful with each generation, but prices stayed roughly the same plus minus inflation. Last couple of years, this trend has broken. You pay double or even quadruple the price for a relatively tenuous increase in performance.
You get the point.
There's always local optimization that leads to improvements. Look at the Apple M1 chip rollout as a prime example of that. Big/Little processors, on die RAM, shared memory with the GPU and Neural Engine, power integration with the OS.
LOTS of things that led to a big leap forward.
You can re-arrange them for minor boosts, double the performance a few times sure, but that's not a sustained improvement month upon month like we have in the past.
As anyone who has ever optimized code will attest, optimization within fixed constraints typically hits diminishing returns very quickly. You have to work harder and harder for every win, and the wins get smaller and smaller.
However, none of that is actually important when the thing people care about most right now is energy consumed per operation.
This metric dominates for anything battery powered for obvious reasons; less obvious to most is that it's also important for data centres where all the components need to be spread out so the air con can keep them from being damaged by their own heat.
I've noticed a few times where people have made unflattering comparisons between AI and cryptocurrency. One of the few that I would agree with is the power requirements are basically "as much as you can".
Because of that:
> double the performance a few times sure, but that's not a sustained improvement month upon month like we have in the past.
"Doubling a few times" is still huge, even if energy efficiency was perfectly tied to feature size.
But as I said before, the maximum limit for energy efficiency is in the order of a billion-fold, not the x900 limit in areal density, and even our own brains (which have the extra cost of being made of living cells that need to stay that way) are an existential proof it's possible to be tens of thousands of times more energy efficient.
Ditto with mobile phones. iPhone may be more expensive than when it launched, but you can buy dirt-cheap chinese smartphones that have similar performance - if not higher to the first iPhones.
Missing the point, despite being internally correct: 10% of $700k/day is still $25M/y.
If you'd instead looked at my point about energy cost per operation, there's room for something like 46,000 improvement just to human level, and 5.3e9 to the Landauer limit.
How is it not?
These LLMs were recently trained using NVidia A100 GPUs.
Now NVidia has H100 GPUs.
The H100 is up to nine times faster for AI training and 30 times faster for inference than the A100.