Its too bad research papers can't be organized like a git history. We'd see many forks that never end up as pull requests that are merged back to main. And probably forks of forks that stray too far from the founding paper's intent. It would be nice to more easily identify original versus derivative research. Maybe that solves a different problem. I like their suggestion though:

"I offer a prag­matic cri­te­rion: what makes a crit­i­cism im­por­tant is how much it could change a re­sult if cor­rected and how much that would then change our de­ci­sions or ac­tions: to what ex­tent it is a “dif­fer­ence which makes a dif­fer­ence”. This is why is­sues of re­search fraud, causal in­fer­ence, or bi­ases yield­ing over­es­ti­mates are uni­ver­sally im­por­tant: be­cause a ‘causal’ ef­fect turn­ing out to be zero ef­fect or grossly over­es­ti­mated will change al­most all de­ci­sions based on such re­search; while on the other hand, other is­sues like mea­sure­ment error or dis­tri­b­u­tional as­sump­tions, which are equally com­mon, are often not im­por­tant: be­cause they typ­i­cally yield much smaller changes in con­clu­sions, and hence de­ci­sions."

So, 2 papers, both with data and claims.

The first is critiqued on its claim because the data, while correct with quality methodology, doesn't support the extent made in the claim. This critique is more meaningful because it changes the outcome of the paper and any decisions following its publication.

The second's claim is within the bounds of the data but there is a discrepancy in the data collection which is the source its its critique. Fixing that doesn't change the claim but may indicate more research is needed. This critique could change decisions made from publishing, but if the claim is still within the reason of the data, then likely not.

I had to think through that and I think I like it.