I also had a few do-overs at the end of my thesis, but fortunately had a cluster standing by...
Well, there's also this notion of testing and regression. As I said in another comment a few days ago:
>A few weeks ago I had a conversation with a friend of mine who is wrapping up his PhD. He pointed out that not one of his colleagues is concerned whether anyone can reproduce their work. They use a home grown simulation suite which only they have access to, and is constantly being updated with the worst software practices you can think of. No one in their team believes that the tool will give the same results they did 4 years ago. The troubling part is, no one sees that as being a problem. They got their papers published, and so the SW did its job.
An independent implementation of the experiment is neccessary for a full reproduction anyways. If you just run their code again, you'll end up with all their bugs again.
(But, don't get me wrong. I like when researchers release their code. It's still very useful.)
1. People will not bother. It took a lot of minds to come up with the software used (in my friend's case, several PhD's amount of work). No one is going to invest that much effort to invent their own software libraries to get it to work.
2. Even when you do write your own version of the software, there are a lot of subtleties involved in, say, computational physics. Choices you make (inadvertently) affect the convergence and accuracy. My producing a software that gives different results could mean I had a bug. It could mean they did. It could mean we both did. Until both our codes are in the open, no one can know.
It is very unlikely that you'll have a case of one group's software giving one result and everyone else's giving another. More like everyone else giving different results.
Case in point:
https://physicstoday.scitation.org/do/10.1063/PT.6.1.2018082...
HN discussion:
If there's one thing I could convey to the world from what I learned from my time in academia, it is this: Most scientists at universities do not care about reproducibility.[1] Not only that, many people intentionally omit details from papers so that it is hard for rivals to reproduce their work - they want the edge so they can publish without competition. This isn't a shadowy conspiracy theory - this is what advisors openly tell their students. Search around on HN and reddit and you'll see people saying it.
[1] My experience is in condensed matter physics - it may not apply to all of academia.
I can believe that. I have seen code in academia and it was all one or two-letter variables without even linebreaks between statements.