Hacker News Reader

It should be a requirement that every paper be completely reproducible, for any software part of the paper. On any OS or systém, it should work. Similar to requiring statistics to be reported a certain way, there should be guarantees that the code should be available and work in all circumstances.

View on HN

Unfortunately, I don't think this is realistic. Is it provable?

Does it require use of an "official" programming language and libraries, that are guaranteed to be maintained? Is Turbo Pascal for MS-DOS still supported? That's what I used for my dissertation, in 1993.

It's a far cry beyond what's expected in branches of science that are not mainly about programming.

Perhaps require making a docker container and writing a test script that includes inputs and expected outputs. There could be a website that loads those docker images and shows that the tests pass. Tests could be referenced within the paper. Would still require scrutiny to make sure that code doesn't hard-code the outputs.

Reply on news.ycombinator.com