Yes, I wish the original, apparently C code, had been released but let's fix the bugs in the code.
The issue claims the tests are broken because they're checking for checksums instead of actual data -- but what they really do is to check for the sanity of the implementation to be actually implementing the mathematical models that are underlying to it.
That's the beauty of open-sourcing the code: people can help or verify. Needlessly shitting on other people's work without any proof is just disheartening to me.
With that said, if a research paper's main contribution is a model whose results were evaluated using a code with major bugs then retraction of the paper is only professional.
There's no proof of that at all in this case. The person complaining didn't find any scientifically valid problem, only that he personally for some not cleanly stated reason doesn't like the tests provided, which is clearly stupid.
Since when should any tests be such that some random guy on github must like them? If the code was used by the experts, and they maybe even used it for years, who says that these experts are in any way obliged to publish all their logs of their use of that code (which doesn't have to even exist in publishable form)?
Even if all that they possibly iteratively did for potentially years could be condensed to some tests, who says that it wouldn't take too long (as in, months, years) for that? It's practically just a matter of good will that the code is published at all, in any form at all.
The overall conclusion may be correct, who knows, but based on my experience, I do not believe, at all, any numerical predictions of this code.
That does lend itself to retracting the paper.
This I take issue with.
Yes, it is a political proclamation without any practical substance, and should be recognized as such.
The 116 is "if I use it this special way I would expect something else and not what I see" which can be simply answered with "well, don't do THAT."
The simulations by design are not expected to produce exactly the same results in different runs. That's why they are simulations. The reporter tried some "partial save" and expected something else.
The 30, if I understand correctly, is observed on Cray to behave in some minor detail not exactly the same as on the PC (which again doesn't have to even mean that the output is scientifically wrong). Seriously? If one is a Cray user, he can fix it, so what?
https://github.com/mrc-ide/covid-sim/issues/168
Looks linked to the awful style, though. When you have 700 LOC functions starting with
int i, j, k, l, m, i1, i2, j2, l2, m2, tn;
Just smells like FORTRAN to me.
Did this model provide clarity and insight for the decision making process? Or, did it instead induce a false sense of panic?
Here's some more info as to what this repo does and does not contain, and why what was released is not enough.
https://www.aier.org/article/imperial-college-model-applied-...