There's not one legal question, but many. If you live in a jurisdiction with different rights by sex-or-gender, which rights do you get? If you live in a jurisdiction with legal heterosexual marriage but illegal homosexual marriage, who may you marry? If you represent yourself as (fe)male to someone who cares, are you committing fraud? When contracts legally discriminate between genders (e.g. maybe scholarships for one gender only?), do you get to choose which gender you are for those contracts?

There're medical questions: if sex is a useful predictor of therapeutic effectiveness, what should you be telling your doctor? What should E.R. techs do? What if _gender_ is a useful predictor of therapeutic effectiveness? (Consider difficult cases like this: someone who is XY but was gender-reassigned in infancy to female (it has happened!), and raised female, and believes emself female.... which pathologies will follow male distributions, which will follow female, which will be something else?)

There're public-space social questions: which washrooms can you enter, which changerooms? (mutatis mutandis for other gendered public spaces). Can you attend girls-night-out? (Some friends-of-friends do a ladies night that includes transwomen; when I've met them, I'm inclined to think of them as dudes. How limited of me.)

And then, arguably most important, there're private social questions. We've already mostly worked out (at least in the left-leaning parts of the West) that neither sex nor gender determines who you want to have sex with, and that's probably fine (I personally think it's 100% fine!), but baby that's just the tip of the complicated iceberg. What do you do if you have an Adam's apple, and big hands, but you also have breasts, and you want to find people to discuss bra-sizing with?

tl;dr: gender and sex are complicated, and there are 2 defaults that seem to mostly-work for maybe 80%-95% of the population, and then a few more semi-defaults that work for maybe 80%-95% of those left, and then a few more defaults that work for maybe 90% of those left, and so on. And this shit matters to people, seriously. Don't be a dumbass by saying "oh it's as simple as X".

However, that doesn't state your gender definitively. Intersexed folks exist, people who were mistakenly mis-assigned gender by their genitals exist, and you can change your legal gender, at least where I live.

Do you mean "legally a man", or "medically a man"? It is trivial to conceive of legal systems that fly in the face of what is plainly reality. The law can, and at various times and places has been, written by bigots and/or the uninformed. Who cares how it classifies things?

As for "medically a man", while it is possible to be "medically a man" or "medically a woman", for any reasonable definition of "medically", there are more than those two options.

Your comment seems to suggest that you don't realize that one's legal gender status can and often does follow one's gender identity: http://en.wikipedia.org/wiki/Legal_aspects_of_transsexualism