Hacker News Reader

Comment by nohuck13 | Hacker News Reader

"It’s the same story, really: It is a software engineer’s job to build quality software. A scientists job is to solve problems."

That's not the distinction. Good software engineers solve problems. That's what the paycheck is for. The distinction is whether code has to be maintained.

It's the scientist's job to solve a specific problem at a specific time. Who cares if the metaphorical wood rots next winter, the paper's been published.

It's _often_ the software engineer's job to build things that deliver business value over years, evolving and expanding requirements, in a development team, without grinding to a halt under the weight of accumulated complexity.

"Quality" software engineering is just heuristics for keeping the pace of change high over time without breaking things.

View on HN

> Who cares if the metaphorical wood rots next winter, the paper's been published.

Isn't this why the replication crisis was able to be kept hidden for so long?

Not as far as I know. Much more problematic was the fact that replication is very hard to publish. That's because either you more or less confirm previous findings and therefore contribute little to the scientific record (or so reviewers seem to think), or your findings counter the original results and now it's on you to explain the discrepancy. Even if you satisfy the reviewers that you're right, they may not consider your result of sufficient caliber to accept for publication in this particular venue [1].

[1] I've seen this happen to a paper that proved that a theoretical framework for constructing proofs about RFID protocols was neither sound nor complete.

> It's the scientist's job to solve a specific problem at a specific time. Who cares if the metaphorical wood rots next winter, the paper's been published.

Sounds a little like cargo-culting than proper reproducible research. But this is a pest in academia definitely. Many papers do not provide all required data, all required model parameters etc. to get to the exact same result. Admittedly, they might nowadays need a software engineer to get that done.

Cargo-culting is about focusing on the process without fully understanding its purpose. Such as bureaucratic requirements for providing all data, software, parameters etc so that someone can reproduce exactly the same numbers with minimal effort.

Proper reproducible research is not like that. It's about providing sufficient details that other people in the field can extrapolate the rest. That they can use similar methods with similar data to achieve similar results.

Reproducing exactly the same result is not that valuable, as the "result" could be just an artifact of the specific data and specific methodology. Real validation depends on fully independent replications, with as little reuse of data and code as reasonably possible.

In software I have found getting the exact details and parameters very useful even if I don't intend to use them. Because when I try to do the same thing my own way and fail, then I can reference the original ones and gradually make my own version more and more similar to that, and see when it starts working. Or make their version more and more similar to mine and see when it breaks. This allows quickly easily identifying the critical difference.

If we cannot reproduce research results, due to missing the data, parameters, or code, or whatever else, is it not cargo culting, to take that research result and blindly believe it and build on top of it? If I remember correctly, Feynman stated in the same video, that one should actually reconstruct or reproduce the experimental results we rely on.

Reply on news.ycombinator.com