It's appropriate to discuss generalities. It's appropriate to discuss individual cases. What is inappropriate is dismissing individual cases to come to a general conclusion, and then insist that it applies universally.
This is also in the context of trans people the vast majority are not biological outliers simply people identifying as something different to their unambiguous biological sex.
To make this claim, you would have had to find an exhaustive list of biological sex differences, and check a large representative random sample of transgender individuals to see whether all of those biological sex traits aligned "unambiguous[ly]" with their assigned birth sex.
Since I am quite confident that this work has not been done, may I suggest deferring to individual people to tell you what is going on in their bodies? While imperfect, self-reporting is still epistemologically far superior to you just pulling this claim out of your ass.