Hacker News Reader

Anecdata seems quite valid for LLM comparison when trying to evaluate 'usefullness' for users. The lmsys chat leaderboard is literally just mass anecdata.

Yes, "mass anecdata" + blind collection is usually called "data".

You might also want to look into other benchmarks: https://old.reddit.com/r/LocalLLaMA/comments/1ean2i6/the_fin...

GPT-4o being only 7 ELO above GPT-4o-mini suggests this is measuring something a lot different than "capabilities".