Of course, it's not like society has any choice - if rival nations are willing to genetically engineer humans, can we afford to fall behind?
I don't see what's so hard about it. When something gets better than it used to be, that's an improvement. Better eyesight, hearing, intelligence, endurance, strength, etc etc - improvements.
I won't get into any argument that "less is more". I don't believe that crap. We shouldn't stop until we find the limits of the universe.
When it comes to differences between people, there are a lot of traits that we should be careful about categorizing as better or worse. Especially if it leads to us essentially eradicating the trait via eugenics.
I don't think we're anywhere near genetic manipulation of human mind, if that's even a real thing. Perhaps we could manipulate behavior driven by hormones, but I don't think we could do too much there without breaking the whole system.
If the negatives subjectively don't outweigh the positives, that's usually considered "deal with the devil" territory and is the kind of thing sci-fi writers love.