>On a personal level, I’d go further and suggest that all academic epidemiology be defunded. This sort of work is best done by the insurance sector.
This is another level of crazy.
>On a personal level, I’d go further and suggest that all academic epidemiology be defunded. This sort of work is best done by the insurance sector.
This is another level of crazy.
No it isn't.
> It's well known that academic software doesn't follow great software engineering practices.
That doesn't make it acceptable. Also, it's also "well known" that vast majority of academic work has zero tangible effect on society. This isn't one of those works. It's possibly the most important piece of academic work that has happened in recent memory. So the bar for this is MUCH higher than typical academic research work.
>. That it isn't fully deterministic (with a fixed random seed) doesn't make thee research invalid and shouldn't discredit research that builds on the model.
It does make it invalid, when the difference between runs is as big as 80,000 estimated deaths which can lead to dramatically different government policies.
> This is another level of crazy.
No it's not. Academia is way behind the industry when it comes to modeling the economy and the real world.
The insurance industry is expected to ask the government for bailouts because none of their models can account for the fallout from this, just like AIG did during the '08 crisis.
Response: An entire industry of competing and well-funded actors that specialize in modeling and predicting expensive failure states also failed to model the world accurately.
Implication of Response: Where is the evidence that this is inferior to industry?
A meta implication is that expert models were insufficient to inform decisions.
Wrong. Nobody decides policy based on whether there will be 320k or 400k deaths. What matters is the order of magnitude.
This difference alone is the difference between deploying the army to build field hospitals and emergency seizures of industry to make it happen, or doing nothing. And it comes from floating point differences between AVX and non-AVX hardware. The apologetics for this on HN are absurd.
That is not relevant.
> This difference alone is the difference between deploying the army to build field hospitals and emergency seizures of industry to make it happen, or doing nothing. And it comes from floating point differences between AVX and non-AVX hardware.
No, 320k or 400k deaths will not make the difference to decide whether the army will be deployed.
If the 80k were the difference between 100 and 80100, then it obviously would impact policy.
What matters is the relative difference, not the absolute. How is this so hard for you to understand?
If you code has race conditions that make your simulation roll out and produce different results, then you aren't measuring the result of the simulation, you are measuring race conditions.
Of course, you could be lucky and race condition could be harmless or contribute almost nothing to the result. But this is unusual, needs proof and indications are it's not the case. Responding to that "well, it's stochastic so what do you want!" is not the way it works.
Poor code quality is a sad reality in many scientific projects, yes. But it's not just a code that is unreadable. It's the code where race conditions knowingly mess with the model results! You can't just dismiss that.
It might be better, perhaps, for an academic to perform a code review instead.
This makes the criticism (although partisan) at least worthy of attention.
Well, know that literally trillions of dollars have been lost, it's not a surprise that the computer model at the origin of the panic is being scrutinized.
So when the work they do is this bad, I think it's reasonable to question whether they should be continued to be paid for no value to the public.
To be fair, if such code would be used for a paper and the flaws came out, I think it would warrant a retraction.
I think I saw not too long ago a story right here where a software error caused completely wrong results.