I used to work with an ex-military guy (extremely bright, hard working, excellent leader, etc. etc.).
He was about 64 inches (1.63 meters) tall, balding, and not the handsomest fella. He told me point blank: "I was never going to become a general or achieve any other extremely high rank simply because I didn't have 'the look' or physical stature of a general."
Compare Canada where, beyond not having photos, even gender-based pronouns have been removed from performance reports in order that reviewers not risk gender bias.
That would be gender-blind but also ignores the fact that absolute scale actually matters in war sometimes - running speed, ability to lift and load heavy tank shells for a few hours etc.
One might argue that actual physical strength doesn't matter much. One way to check this claim would be to look at recently granted medals for valor, and see how many of them are related to absolute (not relative) physical prowess.
Indeed, the higher up you go, the more your job becomes about leading and the less it becomes about individual feats of strength. I think you're thinking about the military a little bit wrong here -- this isn't the Roman era (when centurions led from the front and had to be legitimate physical badasses). Being higher up in the military nowadays is a lot more akin to being a paper pusher director at a company.
Medical and fitness are different things. Lots of people can pass the fitness tests but still flunk out for medical reasons. Diabetics. People who get cancer. Failing eyesight. Sever allergies. It is very possible to ace the fitness tests while still failing the medical.
If there was one, there was at least a ton of exceptions to the rule, male and female.
EDIT: Though I didn't work with any generals. There are only several hundred U.S. generals, and that number is capped.
Take Thatcher or Merkel. Both women for whom this arguably matters even more, not attractive, and not even coming from a background that lends itself naturally to climbing hierarchies. Kohl, Merkel's mentor used to often refer to her as his "Mädchen", ('girl', but diminutive, only used for children) and she basically back-stabbed him and took control of the entire party.
There's definitely ways for unattractive or more generally, people who nobody expects to rise to power to do exactly that.
This New Yorker profile from a few years ago is a long read but very much worth it if you want to get a overview. The beginning recapitulates a lot about her childhood so you might want to skip that part if you're only interested in her career.