Suppose you have two. One's into javascript and dungeons and dragons. The other's not great academically. No good at football either, and they still behave in a slightly awkward way. But great person, all the same.
So, you condemn the latter child to the cesspit that is public life and protect the former? Or think up a better policy?
Someone with abundance in a community of scarcity is a target for bullying. That bullying robs the individual and society of those gifts.
So, yes, we protect the gifted, precisely so their gifts survive to benefit all of society, including the bullies. That's the irony. The gifted want by their nature to share their gifts and do so -- even with the bullies and at their own expense.
The average individual is much less likely to be bullied and thus need less protection.
"It'll all be ok if we keep away from the riff-raff" doesn't do anything but say "we can only really afford to educate X amount of kids". This lot are brick layers.
Of course who am I talking to? I don't know. Might be somebody who thinks gifted means Daddy has a yacht.
Gifted people don't escape bullying by growing up and it has immeasurably costly ramifications for society at large.