With humans, there is only male, female, and for lack of a better term, very rare birth defects. We say that arachnids are "8 legged species" even if due to birth defect a spider is occasionally born with 7 legs. We say a coin flip is a binary "heads" or "tails" even if there is a vanishingly rare chance it could land on its side.
This whole "what about people with intersex disorders" is total sophistry. If it was just people with intersex disorders, this would not be an issue. The issue is people who are very clearly one sex demanding to be treated as the other sex. The issue is publicly teaching children that it is possible to change sex.