We'll still need adults to raise this next generation. So, if we realize we swung the pendulum too far towards uniformity, it'll take 16+ years for the next "diverse" generation to mature and become self-sufficient.

Unfortunately, diseases typically run their course much faster than that meaning it would be possible for a disease to devastate a genetically homogenous population before they can raise the next generation and maintain population levels.