Ideally, what we need is being able to 1) reproduce from scratch, 2) rerun the experiment with the code used (which has a lot of pragmatical advantages too).
In fact, I would argue that even the examples you are giving are not "reproducibility". Reproducibility means that if you redo it _from scratch_, you reach the same conclusion. If the conclusion is "this algorithm describe in this paper is 20+-10% better than the algorithm described in this other paper, in those conditions", then, ideally, what you need to do is to read the first paper and follow the explanation to recreate the algorithm, read the second paper and follow the explanation to recreate the algorithm, read the conditions and reproduce them yourself, and then, you check if you get a compatible conclusion.
As for improving on top of others, again, the scientific paper explains the algorithm, just apply the recipe in your personal implementation, and you will have the same result. Even better: if you don't, you may have found a bug that exists in their implementation, and you have made a scientific progress that you would not have obtained if you would have just cut-and-pasted their code.
But as I've said, the ideal is to have both. Still, the sentence that we reacted on as "not scientific because not compatible with reproducibility" is still valid. It is true that, in practice, the code is not very useful in science (it may be a bit useful, but not as crucial as some may think), and it is true that this does not lead to a reproducibility problem, because the goal of the concept of reproducibility is to guarantee the existence of _independent_ implementations to check if we reach the same conclusion.