This is a statement about social identity rather than biology. Transgender people have little or no interest in opening their gray matter or genitals to public scrutiny. It's about respecting the feelings and choices of the individual.
Consider this analogy: suppose that we refused to call people "Christians" who self-identify as Christian but nonetheless ignore or even act in contraction to the teachings of Jesus Christ? As far as I can tell, this would apply to the majority of so-called Christians in the United States. (I personally refer to these people as "Old Testament Christians".)