It would help America more to have universally "pretty good" schools, than a few great schools in a sea of "pretty bad" schools.
So, like many other American issues, the argument reminds me of two people in a tiny parking lot squabbling angrily over who is at fault for boxing the other in. If the business that designed and painted the lot allocated insufficient space, neither driver should blame the other.
The premise that there are groups in America who need saving from poverty and lousy schools is true, but why save only a few of them by sending them to a school for gifted students? Deal with the inequality of income and of the school system. Last time I checked, Finland does pretty well with that strategy.