As you said if the library/framework provides deterministic computation, then it should not be a problem, I am not sure if it's already fixed in Tensorflow, but parallel computation and order of operations could all influence the reproducibility of the run, so sometimes you need to provide that in your code.

What Polyaxon provides, is a way to restart a run based on as many parameters collected, by default it will use same code based on the internal git commit, it will reuse the same configuration, same Dockerfile, and if provided, it will use same resources, CPU, GPU, and memory of the original run, if the experiment had a node selector, the restarted experiment will be as well scheduled on the same node.