Agreed!

And, maybe I'm missing something, but to me it seems obvious that flat top part of the S curve is going to be somewhere below human ability... because, as you say, of the training data. How on earth could we train an LLM to be smarter than us, when 100% of the material we use to teach it how to think, is human-style thinking?

Maybe if we do a good job, only a little bit below human ability -- and what an accomplishment that would still be!

But still -- that's a far cry from the ideas espoused in articles like this, where AI is just one or two years away from overtaking us.