1. Rapid change of any kind is bad in any ecosystem. Growing the population too fast creates one set of problems, shrinking the population too fast creates a different set of problems.
2. modern humans have the ability to expand consumption to use all available resources. 10 million jet-setting billionaires have a far worse impact on the environment than 1 billion poor people. Adjusting the environmental impact per human is going to have a far greater effect on the environment, and more quickly, than population changes.