This isn't to say we shouldn't think critically about the use and performance of models, but "Not Even Bronze..." turned me off to this critique.
Yes, we're all impressed, but it's time to move on and start looking at where the frontier is and who's on it.
(It's specifically trained on formalized math problems, unlike most LLM, so it's not an apple to apple comparison.)
It wasn’t that long ago that the Turing Test was seen as the gold standard of whether a machine was actually intelligent. LLMs blew past that benchmark a year or two ago and people barely noticed. This might be moving the goalposts, but I see it as a realization that thought and language are less inherently connected than we thought.
So yeah, the fact that they even do this well is pretty amazing, but they sound like they should be doing so much better.
It's not an unfamiliar phenomenon in humans. Look at Malcolm Gladwell.