Why wouldn’t the Down Syndrome patients themselves have children?
Homosapiens are a result of chromosomal fusion, so I would approach the claim of "accruing defects" with great skepticism. Nature is still selecting as evidence by inflicting a high miscarriage & infertility rate.
https://www.ndss.org/resources/sexuality/
https://www.merckmanuals.com/home/children-s-health-issues/c...
That said, it is definitely immoral to forcibly prevent another human from having a child. Any debate on morality must focus on the decision the individual themselves must make.