Complaining about "bad charting" and posting a chart with y-axis that doesn't start at 0 is kinda weird.
there's a fun Excel artifact where it auto-selects the 'relevant' range with no adjustment for how proportionally close to 0 the values are - a professional researcher publishing to a journal should know better (and should be ridiculed for not incorporating best practices) but for a personal blog by an SWE this really isn't the worst sin
[0] https://digitalblog.ons.gov.uk/2016/06/27/does-the-axis-have...
Just look at the first chart: the distance between Fable 5.1 and Sol is <5%, but it looks like 25 or 30%.
And as the complexity of your score increases, the cutoff goes up. We can quibble about where your personal cutoff is, but it aint 0.
But even if it was between 0 and Inf+, it still gives you a wrong perspective, especially if you are not paying attention, on model capabilities.
What is bad is starting at 0, showing an indicator of a gap, and suddenly starting at 30 or whatever after the gap.
If I'm not wrong OAI tried a similar trick when GPT5 was announced ... they have been criticized a lot.
Only if you aren't schooled in reading graphs. It's a given that you always have to look at the axes when interpreting a graph.
How exactly would you zoom into a section of a graph and just show that section?
So we can say the same about the authors "AA’s plot is misleading" claim, he is "not schooled in reading graphs"?
> How exactly would you zoom into a section of a graph and just show that section?
When building a chart is good practice to provide log scale switch and zoom&pan capabilities, so the reader can decide how to look at it.
Oh absolutely - as other commenters have pointed out.
> When building a chart is good practice to provide log scale switch and zoom&pan capabilities, so the reader can decide how to look at it.
For the majority of the time charts have existed, your "good practice" would have been impossible. Charts have historically been static images (e.g. published in a journal). So there have been conventions on how to depict them - and at times it is very appropriate to start from something other than 0.
Here's an article from the UK's Office For National Statistics:
https://digitalblog.ons.gov.uk/2016/06/27/does-the-axis-have...
But that era is gone since decades, nowadays, given how easy it is, it's a shame to not provide log scale switch and zoom&pan capabilities.