"pace at which the models are getting smart is accelerating". The pace is decelerating.
[1] https://emp.lbl.gov/news/new-study-refocuses-learning-curve
[2] https://ourworldindata.org/grapher/solar-pv-prices-vs-cumula...
The deceleration of pace is visible to anyone capable of using Google.
1) the dearth of new (novel) training data. Hence the mad scramble to hoover up, buy, steal, any potentially plausible new sources.
2) diminishing returns of embiggening compute clusters for training LLMs and size of their foundation models.
So there's a tension.
Some concerns that we're nearing the ceiling for training.
While the cost of applications using foundation models (implementing inference engines) is decreasing.
Someone smarter than me will have to provide the slopes of the (misc) learning curves.
If its reasonable, then reason it. Because it is a highly apples to oranges comparison you are making
Lots of benchmarks exist where everyone agrees that higher scores are better, but there's no sense in which going from a score of 400 to 500 is the same progress as going from 600 to 700, or less, or more. They only really have directional validity.
I mean, the scores might correspond to real-world productivity rates in some specific domain, but that just begs the question -- productivity rates on a specific task are not intelligence.