(1) take a 300 line NN algorithm,
(2) throw a quarter of the world's GDP + all literature ever collected at using the algo to train a NN
(3) throw another quarter of the world's GDP at billions of teraflops for inference, and
(4) aim the resulting world's-largest-computer at marketing itself to investors, for instance by decoding ciphers from obscure medieval manuscripts,
that you could perform some pretty magical tricks. There are other feats humans have performed for less cost, like sending people to the moon, or landing a rocket vertically, or idk, curing Polio.
I'm not knocking the "miraculous" advance here. The unique thing about the solution which makes it particularly non-trivial and something that humans would struggle with was exactly what LLMs excel at: Diffing loads of texts against each other. But the 176k tokens at around $10 doesn't tell the story of the cost. It says a lot about the externalized cost and the amount of money flowing in to support the hardware. If they'd put a $100,000 bounty out to solve that cipher, I think the internet would've solved it in a couple days.