That brings up a separate issue that I didn't comment on above: the expectation that the code runs in a completely different development/execution environment (e.g. the one the reviewer is using vs. the one that the researcher used). That means making it run regardless of the OS (Windows/OSX/Linux/...) and hardware (CPU/GPU/TPU, and even within those, which one) the reviewer is using. This would be an extremely difficult if not impossible thing for even a professional software engineer to achieve. It could easily be a full time job. There are daily issues on even the most well-funded projects in machine learning by huge companies (ex: TF, PyTorch) that the latest update doesn't work on GPU X or CUDA version Y or OS Z. It's not a realistic expectation for a researcher even in computer science, let alone researchers in other fields, most of whom are already at the top of the game programming-wise if they would even think to reach for a "script" to automate repetitive data entry tasks etc.

==========

EDIT to reply to BadInformatics' reply below (I'm being rate-limited): I fully agree that a lot of ML code releases could be better about this, and it's even reasonable to expect them to do some of these more basic things like you mention. I don't agree that bit-for-bit reproducibility is a realistic standard that will get us there.