Hacker News Reader

> Are these prominent events? Some of these models are just trained a little longer or cooked with different data? >I think it would be better to list pivotable events, i.e. significant changes in architecture or approach, or a benchmark being surpassed. Otherwise this is just a list of models released by big tech companies irrespective of their importance.

I don't think it's that easy to dismiss changes in recipes, data and training regime, tbh. There have been plenty of "huh, that's strange" moments with LLMs even without major breakthroughs in the overall architecture. The case of one model being really strong at chess while others from the same family (larger to boot) aren't is such a "huh" moment for me. Or the new "reasoning" thing, where it's obvious the new models are better at some tasks than the previous gen. Training steps is another interesting one, where two teams - one fine-tuning on 100k+ samples and one fine-tuning on 1k samples w/ 15 epochs and getting similar performance is also a good example. Or the (extremely readable) simple paper coming out of meta fair ~6 mo ago where they found that repeating some (5-10%) of the samples leads to much better generalisation. All of these are "huh" worthy, and I don't think we thoroughly understand why they happen, or why they sometimes happen and not other times (i.e. w/ 20+T training sets on massively scaled LLMs like the purported gpt5).

I’m familiar with the field of machine learning — I’m not an expert, but it’s quite clear that the developments in recent years have been very significant and very rapid.

There’s actually a tendency among experts in the field to underestimate the power and potential of current AI progress.

As the well known saying goes: 'Garbage in, garbage out.' I believe the same logic applies in reverse when extremely high-quality data goes in, an exceptionally good model comes out. At the end of the day, it's all about data, training time, and all those so-called 'details'that’s exactly what really matters