It's like ricing your Linux distro, sure it's fun to spend that time but don't make the mistake of thinking it's productive, it's just another form of procrastination (or perhaps a hobby to put it more charitably).
I don’t believe that top tier engineers just skip learning things because they might turn out to be dead-ends or incorporated into tools by someone else; in my experience they tend to be extremely interested in things that seem like minutiae to others when working on the bleeding edge, often implementing their own systems just to more fully understand the problem space.
If it’s a day job for someone and they are not ambitious, fine. But we are at hacker news. I would bet 99%+ of top tier software talent could tell you practical experience with ralph loops this year, or a homegrown variety, simply because they are an attempt to solve a very real engineering problem (early exit, shitty code/incorrect responses, poor context window length and capacity), and top tier software people expect more control of their engineering environment, and success using their tools than they’d get by just saying ‘meh, whatever, I don’t get this and I’ll just wait it out.’
At this point I would just pick the one who's "ethics" and user experience you prefer. The difference in performance between these releases has had no impact on the meaningful work one can do with them, unless perhaps they are on the fringes in some domain.
Personally I am trying out the open models cloud hosted, since I am not interested in being rug pulled by the big two providers. They have come a long way, and for all the work I actually trust to an LLM they seem to be sufficient.
Link to direct newsletter subscription: https://importai.substack.com/
Also (shameless self-promo) I publish a 2x weekly blog just to force myself to keep up: https://aimlbling-about.ninerealmlabs.com/treadmill/
New model comes out, has some nice benchmarks, but the subjective experience of actually using it stays the same. Nothing's really blown my mind since.
Feels like the field has stagnated to a point where only the enthusiasts care.
Since then it's just been a cycle of the old model being progressively lobotomised and a "new" one coming out that if you're lucky might be as good as the OG Opus 4.5 for a couple of weeks.
Subjective but as far as I can tell no progress in almost a year, which is a lifetime in 2022-25 LLM timelines
That brief two week period when Opus could eat entire tickets was simultaneously fantastic and a bit alarming
Can't argue with subjective experience, but if there were some tasks that you thought LLMs can't do two years ago, maybe try again today. You might be surprised.