Hacker News Reader

Nice. This ranking basically matches other benchmarks, from what I can tell.

Which implies this would probably also hold for the larger models, which are sadly not included in the leaderboard.

I think it's limited to using base models (non post-trained), because the post-training would skew the logit distribution. There are ways to "coax" post-trained models back into behaving "like" a base model, I wonder if the benchmark could be unofficially updated with those somehow.

Reply on news.ycombinator.com