E.g. the paper is about some new method to approach X. In general, the paper could be valid even without any program implementation whatsoever; but you'll likely might supplement the method description with some experimental evaluation - but the paper is not about the experimental evaluation or the "experimental apparatus" i.e. the code they used. I mean, the paper is likely making some claims about the method as such, not about any single particular implementation of the method including their own. As a part of the main claim "this method seems good and interesting" they're providing some evidence "we tried, and in certain conditions described here applying method X was 10% better than method Y" - but the actual code used to do that is just supplementary material; the code is a tool they used in research, not the result of the research they published. The code is the "telescope" you used to make an observation, but the paper is about the observation, not about the telescope.
It's just as with medicine - we generally accept papers that say "well, we tried this procedure on 100 patients and 73 of them got better" without requiring video evidence of those 73 patients actually getting better; in the same manner, we accept CS papers that say "well, we tried this procedure on 100 datapoints and 73 of them got correct results" and don't require the reviewers to reproduce the experiments and verify if the experimental results aren't falsified; just as reviewers don't try to reproduce experiments in pretty much any other science.