Yes, passports should acknowledge intersex conditions and this would be far easier if there weren’t people with unambiguous sex trying to use that mechanism too.
In daily life, outside of dating perhaps, most people generally don't care about someone's reproductive role. In a public bathroom, most don't really care what's actually between someone's legs so long as the individual looks like they belong. The authorities almost always care about sex only as a visible characteristic for distinguishing people. When you say the common meaning refers to reproductive role, you are probably saying that only because it's a proxy for the aspects of one's sex we actually care about.
This recent obsession that when we talk about someone's sex we obviously mean in a reproductive capacity regardless of context is a totally political fabrication in my opinion, and there's no good reason for a passport agency to prefer that definition. I believe the information on a passport is meant to help identify an individual -- so common-sense sex, i.e. appearance, is most important. (Similarly, requiring trans people to use the public bathroom they don't visually look like they belong in is only going to cause chaos, if they obey such a law.)
[1] https://www.lesswrong.com/posts/4FcxgdvdQP45D6Skg/disguised-...
There are many other situations where sex and its material consequences matter more than gender. Yes, it’s not gametes that make a man run faster, but there’s a clear causal chain between gametes, sexual differentiation pathways, gonads, and testosterone.
Incidentally, it’s a really weird idea that we can’t tell someone’s sex without looking in their pants, or without some advanced scientific analysis. In reality, outward appearance is extremely highly correlated with sex, and humans (and a lot of animals!) have evolved to be experts at detecting sex, sometimes at a great distance. Visual appearance, gait, voice, behavior, and more, are enough clues to not just make a guess at sex, but to do so with very high accuracy. Probably upwards of 99%. Official documentation is there to clarify the 1%.
There are many definitions of sex. Some would say that sex should be defined chromosomally. I think it would be very unhelpful to put an M on a passport for an XY cis woman -- it should be an F, or maybe an X, if X were just a marker to indicate an unusual edge case. If so, shouldn't transgender people at least be afforded an X on their passport?
And then there's what's between your legs. Well, this is surgically alterable to some extent, so do you mean what's currently between your legs, what appears to be your legs, what was between your legs at birth, etc.? Or do you mean reproductive function? Maybe someone might say that a regular surgery isn't sufficient to change sex, but stem-cell-grown genitals do count as changing sex.
Regarding outward appearance (etc.) being correlated with "sex" (however it's been defined) -- that's exactly my point! In fact, because outward appearance is what is important nearly always -- except possibly in dating, reproduction, and sports -- and because there are a plethora of different meanings when sex is defined as "biological truth," I honestly think that outward appearance (etc.) is what most people actually mean when they say someone is a "man" or a "woman." Therefore, sex, in common usage, is appearance (etc.). This matches with how a tomato is a vegetable culinarily (i.e. colloquially), but a fruit botanically (i.e. "biological truth").
(Anyway, all this is really beside the point of this thread. You can disagree with everything I've said -- I'm just trying to argue that the U.S. is presently a difficult destination for transgender travellers.)