But what do you classify an intersex person as when you see them, assuming you don't know this detail? Our social lens is focused to a binary, even if there isn't a definitive binary.
Let's say you have concrete definitions of man and woman (and as many additional categories as you please). You see a person and believe they fall into one category. As it turns out, you are wrong, and they are actually in another category.
That you miscategorized someone has no bearing on the validity (or lack thereof) of your definitions.