Social advantages based on arbitrary phenotypic traits should not be encouraged. White people generally have social advantages given the fact that racism exists.
But nutrition is what's being measured here. People from poorer countries are on-average shorter, and that is largely driven by diet, although childhood illness is also a factor.
The issue isn't height really, it's that people are having their growth stunted. They did not reach the height they would have if they'd gotten better food, better sanitation, etc.
Racism is certainly bad and should be discouraged. Somewhat controversially, apparently, so should heightism.
Or it is simply that you can physically intimidate people to get an upper hand in situations where such an upper-hand shouldn't be given... just because. How noble. What a useful and admirable trait to encourage.
There is that deterrent in place, that social constract. Sure it can be violated, but your argument is along a similar line of folks who advocate for open carry.
And if you are killed? There is a strong appeal to reducing the likelihood you will die.