> Extraordinary claims require extraordinary evidence so see below for the receipts.
Yes, that’s the kind of attitude I want to see in these model releases
Yes, that’s the kind of attitude I want to see in these model releases
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do