Is it not only natural that people move to a city when it is good, and leave when it is not anymore?

I think there's two interpretations of the observation you describe.