There's got to be a better way of saying this. "A policy is good if it keeps being good after the first time you use it" is the gist that I'm getting, but that's way too simplistic to actually tell you any of the value of dynamic programming.