The ethical problem isn't necessarily with the individual or even the family.
Genetic engineering isn't so simple. Genes frequently have multiple functions. You might edit a few genes and give someone a high IQ, but those same set of genes now make them socially inept.
So let's say tons of parents do this. You now have a generation that can't effectively socialize and communicate. How does that affect our species as a whole?
Most of humanity's accomplishments are a direct result of our ability to work together. We may worship the individual, but it's groups of people that actually get things done. So by tweaking that one little tidbit to increase IQ, you completely threw the delicate balance out of whack and sent everybody back to the stone age.
That's just one hypothetical example.
By bringing gene selection purely to the parent level, you're excluding millions of years of evolutionary pressure for the sake of whatever happens to be fashionable in that moment.
The good news is that most genetic modifications will have immediate and nasty consequences. You can dick around with corn and have fields upon fields that go bad and simply pick out the winners ... try doing that with human beings and there will be a huge outcry, very quickly.