Signs of Diminishing Returns with LLMs

It has been some time since the last major step change in LLMs.

The latest round of models are not huge step changes this time around, but more employing certain steps such as CoT.

For open-AI the o1 model adds a new 'thinking' step to get better answers, o1-mini is cheaper and they say çompetitive with o1 in some areas. So the difference between the two models is not that vast.

For Anthropic, claude opus and claude sonnet are not too different. Opus is meant to be the most powerful model yet even on a paid plan the default option is sonnet and Opus is slower and only marginally better at times.

Until something like GPT-5 or Claude Opus 4 is out, for now it seems like the 'throw massive compute' at GPT architectures has been taken already pretty far, and there's no reason to expect transformational changes in capabilities from subsequent models AFAICT.